Image understanding relies heavily on accurate multi-label classification, which has significantly improved with the appearance of deep learning technologies. Researchers from the Department of Software and Information System Engineering Ben-Gurion University of the Negev, Israel evaluated and compared 10 of the most prominent publicly available APIs for deep learning multi-label image classification, i.e. image object classification,  in a best-of-breed challenge.1 These include Imagga API, Watson IBM Visual Recognition API, Clarifai API, Microsoft  Computer Vision API, Wolfram Alpha Image Identification API, Google Cloud Vision API, as well as several open-source frameworks with the capability of image classification, such as Caffe, DeepDetect, OverFeat, and TensorFlow.

We are proud to share the results of this independent study by the Ben-Gurion University team,  as Imagga made it to the top four performing APIs together with Microsoft’s Computer Vision, TensorFlow and IBM’s Visual Recognition.

The evaluation was performed on 1000 images of the Visual Genome benchmark dataset, using 12 well recognized similarity metrics and an additional semantic similarity metric allowed deeper insights for comparison. These metrics evaluate the prediction performance of the APIs based on whether a predicted label exists in the ground truth label or how semantically close it is to it. Three APIs outperformed the rest when evaluating the APIs labels’ predictions with these well-known metrics: Microsoft’s CV, IBM’s, and Imagga’s APIs.

The authors of the paper concluded, that if one is looking for a solution able to handle with high recall and precision a dataset with as many predicted labels as possible, including several which might not relevant (false positives), the Imagga API should be considered as top choice.

Semantic Image Comparison Imagga

Semantic Image Comparison Imagga table 2

Semantic Comparison of State-of-the-Art Deep Learning Methods for Image Multi-Label Classification

Semantic Comparison of State-of-the-Art Deep Learning Methods for Image Multi-Label Classification table 4

Semantic Comparison of State-of-the-Art Deep Learning Methods for Image Multi-Label Classification table 5

 

PAPER REFERENCE

1Adam Kubany, Shimon Ben Ishay, Ruben-sacha Ohayon, Armin Shmilovici, Lior Rokach, Tomer
Doitshman, April 2019. Semantic Comparison of State-of-the-Art Deep Learning Methods for Image Multi-Label Classification.  arXiv:1903.09190v2

Read full paper here

Imagga image recognition API features image tagging, image categorization, smart cropping and visual search. The technology can be delivered as a Cloud API or/and an on-Premise solution.

image analysis