I see several big problems with developing supervised deep learning neural network (NN) classifiers. The accuracy of the of classification depends on the quality/quality of the labeled data and the topology of the network.
Pretrained weights can reduce the amount of time required to train the network significantly. However, problems can crop with matching public domain pre-trained weights when matching the image shape/tensor ordering in the pre-trained weights to the target image size in the CNN training/test/validation sets.
1. Labeling the data. This is a time consuming process and needs to be semi-automated. My company is semi-automating the product image labeling process. We estimate that the time reduction achiieved by the semi-automation labeling system is 2 orders of magnitude. We use Hadoop map/reduce and multiple web applications to label the data. Semi-automation can be applied to many different tasks. If anyone needs to build a semi-automated labeleing system for your company please contact me.
2. The neural network (NN) topology. It takes a long time to train a network with labeled data. Testing out new NN toplogies requires retraining the new NN with the new topology.
3. Cost. Top of the line GPU’s are expensive. I have been told that AWS charges $7/hour for machines with top end GPU’s (8 GPU’s/machine-not sure of the street price for the machine). That is ~$5k/month. Test out 20 different design/training set combinations for a month and you burn $100k on AWS.
As always corrections and comments are appreciated.