Content Moderation - Future Trends & Challenges
During the last few years, content moderation powered by Artificial Intelligence has grown exponentially. It has developed in ways that were unimaginable just a decade ago, breaking concepts and widening the horizons of what’s possible. In this article we'll have a look at the future of content moderation.
Automated content moderation has been fueled by ever-evolving machine-learning algorithms that constantly improve in accuracy and speed. Just 10 years ago, image recognition was only able to classify and detect basic objects and shapes. Now, thanks to the advancements of deep learning, image recognition algorithms for instant detection of all types of inappropriate visual content are a reality.
Automated (also referred to as semi-automated) content moderation thus offers important new capabilities for businesses of different venues that need effective screening of digital content. The AI moderation platforms address a number of key challenges that online platforms and companies face, including:
- Huge amounts of user-generated content need to go online immediately, but still have to be monitored for appropriateness, safety, and legality. This can make it difficult for online platforms to grow and scale internationally if they don’t have an effective way to screen all postings — textual, visual, and even live streaming. Without moderation, these businesses risk great reputational harm, along with a list of other negative consequences.
- Content moderation has to happen in real-time, which is especially difficult for live streaming and video that are becoming the most popular content formats. The complexity of screening visuals, texts and moving images at the same time is tremendous.
- User safety and especially the protection of vulnerable groups is becoming a priority in legislation that covers digital platforms. This means that in many places across the globe, online businesses are required by law to have solid Trust and Safety programs and protection mechanisms based on content moderation. This is necessary not only to ensure the upholding of their internal principles and guidelines, but to safeguard consumers.
- The stress and harmful effects on human content moderators from exposure to shocking, violent and disturbing content is significant. Digital businesses aim to minimise these negative consequences and to protect their moderating teams from the worst content.
- Digital platforms have to be able not only to scale in terms of countries and amounts of content that goes live, but also to adapt to quickly changing circumstances and norms for content appropriateness.
- Public manipulation, political propaganda, disinformation through fake news, and the rise of DeepFakes are disturbing yet prevailing new phenomena online. Both official authorities and online platforms need an effective way to fight them, and machine learning algorithms are the key to that.
Research challenges
While holding great potential and already showing impressive results, there are challenges that AI-powered content moderation is facing.
One of the major issues with which automated content moderation is struggling is recognizing context. Machine learning algorithms can find it difficult to differentiate between subtle cultural and social trends and phenomena. For example, if the algorithm is set to remove all nudity, this is what it would do — even if the nudity is related to art or important news pieces. A prominent example was the case from 2016 when Facebook removed the photo of the iconic Vietnamese ‘napalm girl’ who is naked.
Another important challenge that AI platforms need to overcome is multilingual moderation. While they are getting better at it and are surely improving how content moderation in different languages is conducted, there are still obstacles on the way. The process is not only about acknowledging the direct meaning of words and phrases, but their social and cultural connotations that may make them offensive and inappropriate. In this respect, the more feedback machine learning algorithms receive, the better they can become at spotting the nuances in content — which is definitely not a mission impossible, but simply a gradual process that takes processing large amounts of data.
Live streaming and live video are another interesting challenge for AI-based content moderation platforms. They generate such a substantial amount of data per second that manual moderation is simply an impossible task. Moreover, applying AI on each frame of the live broadcast generates high platform costs. A fast and accurate AI needs to be developed to overcome these hurdles to efficient and cost-effective moderation of live streaming and videos.
Societal and Media Industry Drivers
Merry is a fact-checking manager at an online newspaper with a solid and long-standing reputation for trustworthiness. Her job is to ensure every piece of information that gets published through the media’s channels is accurate. She needs to check not only facts and textual references, but also visuals. In a way, Merry is a modern-day fighter against disinformation — the plague of today’s digital world.
But doing all of this on her own — and even with her fact-checking team — is a monstrous task. The process simply takes enormous amounts of time and effort. That’s why Merry needs a viable and scalable solution to checking accuracy and preventing the spread of fake news and visuals.
This is especially important during the current national election. Politicians, ordinary citizens, and people who want to spread fake news are all posting about the topic constantly — and it’s very difficult to sift through what’s true and what’s not. In particular, a scandalous story about financial misdealings that one of the party leader’s is involved in is spreading online — mostly fueled by his political opponents. Merry has to figure out how true the lead is and whether to publish it.
With the help of an AI-powered content moderation platform, Merry can screen various materials around the topic for authenticity. She can catch textual references, as well as photos that have already been posted, for example.
Daniel is a content editing manager at an online news outlet. He’s in charge of guaranteeing that all published content complies with the standards of the media and the legal framework. His most challenging task is to ensure the compliance of live streamings. Catching inaccurate and harmful video content in real time is a tough nut to track. Without technical support, it is a burdensome task to monitor live streaming content as it occurs.
This is where a content moderation platform based on machine learning algorithms can kick in. It processes all live video streamings, checking them for inappropriate verbal and visual elements. If there are such, the platform can immediately signal to the editors.
Daniel can test the capabilities of the AI platform in practice during an important live streaming with a local politician at a public rally. The situation is uncontrollable, as it’s a place full of people where anyone can appear and take over the stage. With the help of the moderation platform, Daniel can have the live streaming screened for problematic content throughout the whole event.
Future Trends for the Media Sector
The role of content moderation in the media sector cannot be overestimated — in fact, it’s crucial for its wholesome development on a couple of levels:
- As illustrated in the vignette, content moderation algorithms are the key to fighting the widely spread online disinformation. They can spot inaccuracies in textual data and fake or old visuals and videos. These capabilities can be a gamechanger for the media sector that direly needs adequate fact-checking in the oceans of information that get published.
- On the basis of the massive volumes of content that machine learning algorithms process, they can also make trend predictions about the types of content that needs to be moderated. This can be of huge importance for combating harmful tendencies in user-generated content sharing.
- Content moderation, if done in-depth, can help in detecting the intent of disinformation in order to differentiate between positive (for example, to keep state secrets) and negative (to influence opinions and harm society).
Goals for Next 10 or 20 Years
The long-term vision for AI-powered content moderation is a truly ambitious one.
First and foremost, content moderation would need to leave the semi-automatic status it currently has. To be fully useful, scalable, and powerful, it should be more autonomous. The vision is that AI-powered content moderation platforms would become a monitor that is always on and oversees any and all content that goes online. It would screen for all types of abnormalities to ensure protection of users from harmful and illegal content and a safe online environment without fake news — thus taking care of everything from violence and nudity to propaganda, radicalization and disinformation, and all that’s in between.
The second big goal for content moderation’s evolution is self-learning — which is already in motion, but can reach new heights. With the data that is being fed in the moderation platform in real time, the machine learning algorithm becomes better and better. It expands its knowledge base with practical examples and input from moderators. With time, this is how the AI can become more independent from humans in terms of feedback loop. In the foreseeable future, this can reach a point where the moderation platform becomes an autonomous machine that identifies and filters content accurately and effectively with no human input.
A third long-term goal for content moderation is the creation of instant and efficient on-device moderation. Nowadays, moderation is executed on the server end, only after a piece of content has already gone live. This means that harmful content can sneak in for a moment and be accidentally shown to users. In the near future, moderation would be possible on the customer device itself. This would happen before the content has gone live. This advancement would enable the prevention of illegal and disturbing content appearing on the device level, thus ensuring full protection for end users.
Imagga to be part of CounteR – countering radicalisation for a safer world
We are happy to announce that Imagga is taking part in a consortium working on the critical topics concerning democracy - identifying radical content online and creating an integrated situational awareness system to counter violent terrorism and enable crime prediction.
The CounteR solution, an open platform for analysis and early alert, will be launched under the project. The platform will collect and analyze data from dispersed sources in order to predict critical communities at risk of radicalization and violent extremism and aid law enforcement to detect radicalization.
Radicalization is one of the significant challenges online for societies in Europe and beyond. It’s easier than before for people to be misinformed or exposed online to extreme social, religious, or political ideas. Especially vulnerable are some groups with unmet psychological needs for belonging, status, or such with mental illness. The ease of communication and reaching out to multitudes of possible prospects via social media platforms make this threat imminent. Terrorist groups that use social media for radicalization have mastered their tools – it is getting harder to detect such efforts due to the complexity and mass spread of such problematic content. Early detection and flagging problematic content might be one of the most essential tasks in safeguarding Europe and its democratic societies. A combined effort in adopting anti-radicalization policies, technologies, and tools is becoming super important in the fight to counter violent extremism online.
To address this imminent problem, the CountR consortium will work on a platform that will combine critical technologies and know-how to make it possible for real-time detection of propaganda and radicalization in user-generated content. The various participating partner technologies will achieve the robustness of the detection methods. Imagga will contribute to our advanced image and video image analysis and efficient live-stream Content Moderation, which, combined and cross-referred with text analysis, will make it possible to detect radical messaging even though each media type might not have that particular message.
Imagga’s contribution to the CounteR project is the development of instruments and deep learning algorithms to analyze images used for the automatic detection of radicalization and extremist content. Imagga’s image recognition expertise and proprietary technology for Content Moderation will be used to build robust mechanisms for detecting problematic content. This includes detecting specific keywords, a scene in an image or an action in a video that might be related to or inclined to radicalization.
A year in pandemic: 2020 Imagga recap
2020 was not the typical business year. With the announcement of COVID-19 pandemic in the beginning of March 2020, our business and the lives and business of many others have been transformed. In a way it restarted in a very wired way our plans, activities and work practice. March 2021 marks one year from the announcement of the pandemic and we felt now is more appropriate time to reflect on what have happened one year surviving the pandemic.
COVID-19 changed a lot in our personal lives and impacted our company, and our partners. We all had great experiences but the world pandemic, unpredictable and unexpected, stormed into our lives. Historically, with difficult times comes much innovation. The good news is we are built to stay and we adapt and change, learn from the hardship and end evolve our lives, product and services to better match the new reality.
The following is a recap of Imagga's progress since the turning point in March 2020 when we realized we are operating in the reality of world pandemic.
Kelvin Health Spin-Off
Triggered by the COVID-19 pandemic, we joined forces with outside experts. We started together a spin-off called Kelvin Health that applies Imagga's AI technology to aid the screening and monitoring of medical conditions with the help of thermal imaging. A group of enthusiastic medical doctors proposed to validate their observations that AI-enabled analysis of thermal images of the chest and back can help better understand the development of COVID-19 and its complications. We signed agreements for data collection with several prestigious hospitals in Europe. This is helping us to significantly improve the analytical part of the solution enabled by Imagga's machine learning tech.
The scope of the project went far beyond just COVID-19 and we are currently working on helping monitor and screen various cases of breast cancer. Soon to be announced partnership with a pharma company that is at the forefront of fighting cancer worldwide will help us gather valuable data to aid the AI analysis.
Core Technology Advancement
During last year one of our priorities was Imagga's Content Management Platform. We released a private beta version of the CM platform that offers automatic filtering of unsafe content in images, videos, and live streaming. We've put significant effort into improving the dataset quality, increasing accuracy. Thanks to pilots with several significant customers, we managed to improve the platform's stability and extend it to cater to the exponentially growing demand for live streaming moderation.
Our software stack and deployment strategies got a significant upgrade last year. We've improved the overall performance, including a better migration process, optimized and more stable software stack. This resulted in even better performance on our Cloud API and on-premise solution, increasing their stability.
EU Support for Technology Advancement
2020 put an end to a fascinating project, funded by the EU: Photobox - personal photo organization enhanced by Imagga's image recognition and image classification technologies. We've managed to significantly improve our image datasets and make the organization of personal photos more precise and accurate. We will be integrating Photobox technology into our flagship products, the Tagging and Facial Recognition API’s, to make them even more powerful and exciting for our variety of customers.
We've started a collaboration with ZAZ Ventures, leading deep-tech innovation grant consultants. Creating an excellent grant proposal sometimes goes beyond the workforce abilities of technology startups. It often requires tapping into an elaborate grant proposal framework and/or participating in a consortium of business partners working on a multi-industry solution.
We've got the Horizon2020 Seal of Excellence. Imagga is one of the two companies in Bulgaria for the past two years being recognized with that recognition.
2020 also brought two new EU Commission-funded projects to us. AI4Media is a high-impact project run by a consortium of 30 renowned academia, research, and enterprise organizations. AI4Media, which is funded with a total budget of 12M EUR by the European Union's Horizon 2020 Research and Innovation Programme, will focus on delivering the next generation of core AI advances to help media meet the constantly growing requirements but also comply with the core EU values of trustworthiness and ethical engagement.
The second successfully funded project that we are part of is CounteR. CounteR brings data from dispersed sources into analysis to create an early alert platform for data mining and prediction of and support the fight against radicalization and prevent future terrorist attacks in the European Union. Project partners are several police agencies and NGOs and academies doing in-depth research on public violence and terrorism prevention.
Customer Integrations
Somebody once said there's no better investment than a paying customer. Nothing makes us happier at Imagga but to see the needs of our customers met, enabling them to automate "things that are impossible to do manually."
One of the most significant integration achievements of 2020 is the plant lense of the Plantsnap app right into the interface of Snapchat. That was made possible thanks to Imagga's very high-performing deep learning cloud API meeting Snapchat's demands.
Eden Photos is another success story of 2020 - bringing the power of image tagging to Mac and Windows desktops. Imagga's tagging and categorization APIs enable Eden Photos to provide automated tagging and organization capabilities to your personal photo collection.
Imagga processed 600M images in the last 12 months via it's cloud API and even more on-premise in helping our customers improve their end-user experience, monetize their visual content, or assist them in building custom solutions based on visual content.
Last but not least, having a strong team is the best asset every company can have, especially in challenging years such as 2020. Team of people who are ready to fight adversity to support our common mission and keep conquering new horizons together in the long run. Efforts of the team did not go unnoticed. Georgi Kostadinov, our Head of AI, was featured in Forbes “30 under 30” in Bulgaria for 2020.
2020 was challenging but at the same time very exciting for Imagga, and we believe for many businesses worldwide. It brought both the stress and the freedom coming from uncertainty and marked the efforts we need to put our attention in - fighting for a better, more secure future, and Imagga's technologies are a big part of it. We are excited about how 2021 is already unrolling! The best is yet to come!
Research consortium with 30 leading European organizations including Imagga, ITI, DW and IBM was granted 12M EUR funding to improve the use of AI for media in the next years
We are happy to announce Imagga is taking part in a high-impact project run by a consortium of 30 renowned academia, research, and enterprise organizations. AI4Media, which is funded with a total budget of 12M EUR by European Union's Horizon 2020 Research and Innovation Programme, will focus on delivering the next generation of core AI advances to help media meet the constantly growing requirements but also comply with the core EU values of trustworthiness and ethical engagement.
Currently, the European AI landscape in the field of media is quite fragmented. AI4Media aims at establishing a networking infrastructure to foster deeper and long-running interactions between academia and industry. A big portion of the project will be dedicated to media AI research and ways to implement the findings in the currently available on the market media solutions.
In the last 2 years, Imagga has packaged many of its AI technologies into a software solution for telcos, media companies, and platforms. We have built a solution for the automated organization of very large collections and archives of photos and videos.
Through the AI4Media project, Imagga will build onto that and create a solution that can automatically (re-) organize photo and video content typically used by media companies.
Along with the photo/video content organization for media, we are already busy working on a demonstrator capable of utilizing Imagga’s AI for media content moderation, helping companies and organizations comply with the strict safety and privacy regulations and serve content that’s appropriate for the public eye. By analyzing content, Imagga's technology can flag NSFW (“not safe for work”) content to easily identify problematic pictures and create automated filtering mechanisms, crucial for platforms that rely heavily on user-generated content.
https://youtu.be/yzle4v6_pf0
AI4Media project aspires to widen the network of researchers across Europe and beyond, to foster the development and adoption of AI technologies for media solutions. At Imagga we are thrilled to be part of this exciting collaboration. We strongly believe the outcomes will have a significant impact on the competitive advantage of the European AI industry and especially the media sector.
Response to COVID-19 Outbreak
The world as we know it has significantly changed. We all planned for a good and prosperous 2020 year, but the coronavirus pandemic has shifted everybody’s focus from economic growth and prosperity to keep what we value the most - our lives and the lives of our dear ones.
At Imagga, we are taking the COVID19 pandemic seriously and have taken actions to safeguard our team members and their families. By taking care of ourselves, we also want to make sure we will be able to care about our valuable customers: being able to respond in a timely manner and support them whenever they need help.
In challenging times like that, we’ve taken the following measures to make sure all Imagga staff is safe and operational:
-
Home office.
We’ve always been a location-distributed company, and half of our team works remotely. We’ve canceled all in-person meetings for the foreseeable future and introduced a strict work-from-home policy. Most of our operational meetings so far have been online, so we are well adjusted to the new norm.
-
Getting through it all together.
We encourage all team members to stay in touch with each other in this challenging time for us and our society. We are aware you and your team might be forced to work from home too. We will be here to support you by shortening our response time and providing you assistance in any technical issues you might be facing regarding our APIs.
-
Supporting local communities and global fight with COVID19.
If you are part of a team researching COVID19 and its effect on society, we will be happy to provide you with assistance for Imagga’s AI custom training and free access* to our APIs. We will be happy to run jobs on our amazingly powerful in-house NVIDIA DGX Station and help you build fast a custom AI model addressing the COVID19 challenges.
* Note: For free access to our APIs, custom training, and any other technical issues, please, contact us via email with details of the project, expected usage, your university, or non-profit research organization.
Imagga 2019 Rundown and Plans for 2020
What a year! 2019 was nothing less than super dynamic for us, and we feel super energized and in perfect shape to carry the momentum into the rest of 2020!
First of all, we want to thank all of you who have been with us on this journey - making image recognition work for businesses big and small. Image AI is not just a hype, but something very practical - we help cut costs and save time, making processes more efficient and opening new business opportunities.
We are proud to share with you a summary of what we were able to achieve together in 2019.
New Releases
We managed to develop further and publicly release multiple technology offerings that were in the makings and only accessible to a closed circle of customers.
Visual Search
Through Visual Similarity Search API, you can now implement image-based search capabilities into your software systems and applications. Fashion retail, e-commerce, and household goods leaders are using it to make product discovery easier, offer product recommendations and alternatives to products out of stock.
Face Recognition API
We are excited to share that our Face Recognition API has been recently released to allow companies to unlock precise and robust face detection and recognition functionality in their applications and systems. Besides the obvious need for facial recognition in security applications, facial recognition for marketing and retail has been successful.
Object Localization
As part of our Custom Model Training, you can now take advantage of positional detection of whatever type of objects you need. This approach has been applied successfully in cases ranging from detecting guns in images to finding defects (each defect virtually being "an object") in Industry 4.0 content to AR in mobile gaming.
Edge Deployment
We successfully ported our classification and object detection technologies to run smoothly and efficiently on the edge. Currently, we support the two major mobile ecosystems - iOS and Android. We also started offering a unique hybrid model where part of the image analysis happens on the edge, and the real, heavy-lifting processing is done in the cloud.
Wordroom - Lightroom Classic Plugin
Last but not least, we managed to release a significant update of one of our consumer-oriented "pet projects gone popular" - Wordoom. It's a plugin for Adobe Lightroom Classic that automatically suggests up to 30 keywords per photo and ingests them in the image metadata. It got featured on DPReview, resulting in lots of positive feedback and suggestions for features. We published Wordroom's public roadmap to track all the ideas and help our users see when new features will be released.
New Developments
We started two significant projects aiming to deliver to our customers even better solutions built on top of Imagga's AI technologies.
Content Moderation Platform
One of the most significant developments we ignited in 2019 was our Full-Circle Content Moderation Platform. If your project operates with user-generated content, you most certainly need to monitor it and remove illegal and abusive pieces to guarantee the safety of your users and protect the trust in your brand. Our solution handles every aspect of the content moderation process, providing automatic filtering of unsafe content in images, videos, and live streaming. Read about its capabilities and let us know if you have any feature requests.
On-Premise Ver. 2
Our on-premise solution has always been a critical differentiator to cloud-only offerings. We started a major update of its internals to allow even more comfortable handling of scale when you need to use your infrastructure. Stay tuned for the upcoming release.
Signature Customers
PlantSnap
We successfully continued our long-term partnership with PlantSnap, the best plant identification application on earth powered by our technology to recognize more than 100,000 different plant species. PlantSnap is currently fundraising and on the verge of several huge partnerships, so make sure to take a seat on the rocket ship.
World-Class Game Development Studios
AI is getting deeply integrated into any aspect of the digital world, and gaming is not an exception. We signed several major customers utilizing our technology for use-cases ranging from on-device object detection for AR experiences to internal assets organization.
Wrist Watch Super Brand
Traditional industries may also benefit from image recognition, and we helped one of the most reputable brands in the world to classify wristwatches in an automated fashion.
Key Partnerships
Several partnerships played a crucial role in our success this year and felt nothing but gratitude to the companies
Humans in the Loop (HITL)
Thanks to our close partnership with HITL, now we can handle the whole process after the problem definition, including data collection and data annotation. This way, we close the circle of AI services that aid customers interested in our already established ML custom training, optimization, and deployment.
L3C
We started a collaboration with L3C to offer full hardware+software solution to the UK and European customers, taking advantage of the robust IBM POWER 9 architecture. It turned out it's very well suited for massive model support because of its hardware topology that enables fast communication between CPU and GPUs.
Eden Tech Labs
We started working with Eden Tech Labs, a mobile dev studio, as a close partner and integrator of our technology in various mobile and web platforms, solutions, and apps.
Experiments
IBM Power 9
We did evaluation and experimentation with large model support for semantic segmentation of cityscape and waste images benchmarking IBM Power AC922 vs. NVIDIA DGX Station vs. Amazon Web Services p3.8xlarge instance. Read about the findings of the benchmarking test here.
Funding
We were able to utilize funding coming as a blend of European and local government grants for innovative projects to develop further our solution for better organization of personal photos with code name PhotoBox. If you already use (or plan to use) our technologies for that purpose, please let us know, and we will be happy to discuss the improvements there.
Events
Throughout the course of the year, Imagga took part in various trade shows, industry events, and workshops.
EU Gateway trade mission to Singapore and Thailand
One of the advantages of being an EU based company is that the European Union is providing substantial support in promoting and exporting innovative technologies like ours. EU Gateway trade mission made it possible to meet business partners in Singapore and Thailand.
ML DAY Munich
Georgi did a presentation on the practical applications of AI in Retail and Content Moderation.
BAPLA FOCUS London
Firechat discussion on the state of the art and future of AI in stock and media licensing along with panelist form Getty Images
DMLA Los Angeles
Georgi participated in the event and discussed the state of the art and future of AI in stock and media licensing along with panelists from IBM, Google, and CloudSight.
IBM Think London
Together with our partners L3C, we presented Intel POWER 9 experiment for business applications.
DAM Helsinki
We participated in the Digital Asset Management conference in the heart of beautiful Helsinki, where we presented to the industry how Image recognition and AI can improve workflow efficiency in DAM systems.
Team Expansion
A few things make a company happier than bringing more bright people on board. This year we were lucky enough to extend the team with experienced marketing professionals and young talent in the area of machine learning:
Ina Toncheva, Head of Marketing
Ina has 12+ years of experience in communicating and driving tech product growth in the digital environment for companies like Telerik, Progress, Chaos Group, and now Imagga. Her specialties include marketing strategy, content, product, and growth marketing. Over the years, she has launched numerous software products that have successfully penetrated new markets.
Petko Ditchev, Senior Machine Learning Researcher and Developer
Petko is a Senior Researcher at Imagga. He is a medical doctor turned developer. His focus is on cognitive science and its applicability in the field of AI.
Mihail Mladenov, Machine Learning Developer and Data Scientist
Mahial is a software engineer and data scientist. He joined the Imagga team in June 2019. He is currently working on the practical implementation of A.I in the company projects and supporting them with extensive data analysis.
Ralitsa Golemanova, Senior Content Creator
Ralitsa Golemanova is an experienced content creator and writes for Imagga's blog regularly. She has created content and marketing campaigns for various types of businesses from across the world.
Recognition
It's rewarding when all the hard work a company does is recognized. We were proud to have our image auto-tagging technology ranked as one of the most precise out-of-the-box AI solutions worldwide outperforming competitors like Google Cloud Vision and Amazon Rekognition based on an independent technology evaluation by a reputable Israeli university. What is impressive is that we managed to be among the top performers in precision while at the same time being the absolute champion in recall! A trade-off that is not trivial to achieve.
Plans And Focus For 2020
We already started executing our ambitious plans for the new year with some very exciting updates to the core image recognition technology and AI solutions we are working on to release.
Content Moderation Platform
Content Moderation was a hot topic last year with the apparent need for disruption. Currently, hundreds of thousands of human moderators have been working tirelessly to make our kids safe and prevent misinformation. Unfortunately, they are paying a high price sacrificing their mental health. Imagga's Content Moderation Platform is designed to get the advantage of our NSFW AI capabilities and aid human moderators in AI-enabled filtering out questionable content.
PhotoBox
We are busy building a personal photo organization solution for cloud platforms and telcos that will help the burden of organizing and later discovering visual content.
We continue to dedicate efforts to better adoption of our existing technologies, focusing on Digital Asset Management, e-commerce, and factory management (blog post on our progress in this field), offering custom models and classifiers for various existing and new clients.
As always, we will be pushing significant improvements on Imagga's Core AI platform and also plan an update of the business model.
If you have read so far, you might be a faithful Imagga supporter! We are excited about the opportunities 2020 offers and hope you will join us one way on another in our journey.
Metals Hackathon in Vienna and Industry 4.0 Opportunities for Image Recognition
If you read the technology news of the day, it looks like everybody is talking very seriously about AI. The awareness this creates is excellent, but it’s even better when companies from more traditional industries take up the task to experiment and find ways to apply AI into their daily operations. Companies from medical & health industries, metallurgy are finding ways to use AI and regain a competitive edge.
We’ve participated in Digital Metals AI Hackathon organized in Vienna by Pioneers.io and accepted the challenge to build an exciting new solution for AI-enabled measurement of metal parts. The goal of the hackathon is to generate the best ideas and create new solutions in a rapid prototyping format.
Voestalpine and Primetals were the hosts of the event. By bringing together industry leaders and the best technology leaders from the startup world, the two industrial companies hoped to solve specific real-life challenges and identify potential collaboration opportunities for the future.
There were four predefined challenges that the participating startups need to address. We took part in one of the cases of Voestalpine: Measuring Steel Pieces with a mobile app. The Metals Division of Voestalpine, the market leader for tool steel, is focused on producing and processing high-performance materials. High-speed steel, valve steel, as well as powder materials, nickel-based alloys, titanium, and components, are produced using additive manufacturing technologies.
The goal of the challenge is to develop a mobile app that uses the camera for dimension-measuring for steel parts. The app has to be able to measure the three dimensions of rectangular Voestalpine steel pieces using a photo or video. The use of an app will make the process more effective and less prone to errors.
The way measurements are currently conducted is by measuring three dimensions with a tape measure and manually typing them in via a keyboard. It turns out to be time-consuming, leading to mistakes and causing frustration. The metal pieces that need to be measured are quite heavy and impossible to be moved around for better access or more convenient measurements.
The atmosphere at the event was excellent, with funny ice-breakers, cozy, working areas, and mentors form the metal industry companies present at all times.
We were able to build an impressive rectangular steel piece measuring MVP using the Apple AR Kit, apply computer vision and computational geometry know-how to recognize markers in 3D space in less than a day.
As demonstrated in the short video below, once the markers (we used parts of protein bars packaging as a set of improvised markers ;)) are positioned around the steel piece, the only thing the operator needs to do is to point the camera to each of them until it’s detected and then move to the next one. Once all three markers have been visible (sequentially, not needed to be at the same time as it’s not practical for long pieces), the system suggests the dimensions of the items in millimeters.Of course, the operator may go in manual mode and adjust the measurement if needed.
https://youtu.be/7Wn_l9aZdkY
This whole prototype was built in less than a day after we found out the first evening that a solution trying to have all markers visible in a single shot won’t be practical for a long piece.
Our solution turned out to be promising. Some of the participants and judges couldn’t believe it might work even without full visibility of the three markers.
We are exploring more practical applications of image recognition for smart factories. We believe we have just scratched the surface when it comes to automation and augmentations possible with current AI technologies.
Below is a video demonstrating how our visual search technology can be used for recognizing different machine parts, something of enormous value when some repairing or replacement is needed, and there is a vast catalog of possible inventory.
https://youtu.be/z95S2Ga2jiU
Press Release: Imagga Launches the First On-Premise Solution
Imagga launches the first ever on-premise solution for visual A.I.
San Jose, CA. May 10, 2017. Imagga, a premiere PaaS provider of visual recognition now offers its image intelligence solution for deployment on private, local servers. Companies whose data cannot be shared in the cloud for various reasons can now fully benefit from Imagga’s award-winning content recognition technology.
With the introduction of its on-premise solution, Imagga breaks a new innovation barrier. Companies can now have access to best of breed deep-learning content recognition technology without ceding confidentiality of their data. While Imagga’s cloud-based solution already offers extremely high level of security and privacy, its on-premise solution goes a step further, for when data just cannot leave local servers.
“Our on-premise offering is the ultimate data control solution for companies like private cloud, law enforcement, medical, legal, telecom and highly sensitive corporate content. Installed on a local server, Imagga’s visual content recognition can process billions of images, all without leaving the company’s premises, “ says CEO and co-founder Georgi Kadrev. ”It delivers the highest degree of data security and privacy. No other image A.I. company in the world offers this level of control and security.”
Swisscom, an advanced user of the Imagga’s on-premise solution, regularly processes huge volume of images on their own servers: “Imagga impressed us with the quality of the recognition technology, recognizing both objects and broader scenic categories,” says Andreas Breitenmoser, Project Manager at Swisscom A.G.
Imagga’s on-premise solution can be deployed quickly and seamlessly on any servers with the appropriate technical requirements. The company has been working closely with NVIDIA, the leader in GPU high-performance computing, to optimize all its software to run smoothly and efficiently on NVIDIA GPU servers with Tesla and Pascal architectures.
Highly scalable, the on-premise solution delivers the same level of accuracy as the existing cloud offering, currently deployed by over 9,000 developers worldwide and implemented by companies like Cloudinary, Unsplash, Tavisca, KIA Motors, and Plex TV to process several billion photos and videos in the last year alone.
About Imagga
Imagga (https://www.imagga.com/) is an Image Recognition Platform-as-a-Service providing Image Tagging APIs for developers & businesses to build scalable, image and video intensive apps. The company has been recognized by IDC as one of the top innovators in its space for the year 2016. Built for scalability and easily deployed it offers auto-tagging, auto-categorization, smart-cropping, color analysis, content monitoring as well as custom classification.
For more information, please contact:
Georgi Kadrev
georgi@imagga.com
Paul Melcher
paul@imagga.com
+1 917 304 3875
Media Files
Imagga logo - https://goo.gl/hgqBkE
Georgi Kadrev, CEO photo - https://goo.gl/BCgj43
소개 / 기술 개요
Imagga는 개발자와 기업을 위한 사진 및 비디오를 자동으로 태깅 하는 클라우드 API 플랫폼을 제공하는 업체입니다 . Imagga의 기술은 기업이 보유한 대규모의 사진 혹은 비디오를 능동적으로 분석해 분류 및 관리를 할 수 있습니다.
https://youtu.be/XoZAW3XXgDc
Imagga는 머신 러닝과 CNN(Convolutional Neural Network 컨볼루션 신경망) 알고리즘을 통해 이미지를 정교하게 분석하고, 최적의 검색 결과를 도출하는 솔루션을 개발합니다. Imagga의 API를 이용하면 기업은 자신들의 이미지와 비디오 컬렉션을 분류하고 이로 인해 시각 컨텐츠를 현금화 할 수 있습니다. 또한, 이미지에 주석을 달 필요 없이 이미지 키워드 검색이 가능해져 사용자 경험이 향상됩니다.
기술 하이라이트
- 딥 러닝을 통한 이미지 자동 태깅 & 분류 (CNN)
- 클라우드 또는 온프레미스 기반의 RESTful API 솔루션
- 개인의 이미지 세트/카테고리로 교육 가능한 시스템 ( 커스텀 트레이닝)
- 대용량의 시각 콘텐츠를 분석 할 수 있는 솔루션
- 50개 이상의 언어 지원(한국어 포함)
견적서를 받거나 혹은 미팅을 하시려면 연락 주십시오.
Imagga의 이미지 태깅 기술로 무엇을 할 수 있나요?
- 한눈에 놀라운 정확도와 완벽한 자동화된 대용량의 사진을 태깅
- 이미지 와 비디오 카테고리로 정리 - 알고리즘을 개인의 고유 데이터셋으로 교육을 시킬 수 있습니다 - 커스텀 트레이닝을 통해 개인 데이터 셋을 관리 할 수 있습니다
- 개인 소유 이미지 콘텐츠를 이용해 사용자 프로파일링을 할 수 있습니다.
- 이미지 검색을 향상 시킬 수 있습니다 - 컨텐츠에 키워드를 더할 수 있습니다.
태깅 API
Imagga 태깅 API는 사진의 픽셀을 분석해 물체를 탐지 하거나 특징을 추출 해 낼 수 있습니다. 사진을 대표하는 물체와 컨셉을 분석해 20가지 이상의 키워드를 추천해 줍니다. 여러 기업들은 이 키워드를 여러가지 방법으로 활용하고 있습니다. 그 예로, 사진 검색 기능을 추가 하고, 사용자 프로파일링을 하고, 트렌드를 분석 하는 방식으로 사용한 사례가 있습니다.
자동 태깅 API를 경험해 보시고 싶으시면 아래 링크로 접속 해 주세요
카테고리 API
카테고리API는 키워드 보다는 큰 분류 자로 사진을 분류 할 수 있는 기술 입니다. 모든 사진을 자동으로 정리 해 더 편리하게 사진을 찾을 수 있는 방법을 제공합니다. 개인 클라우드 저장소같은 곳에 저장해 놓은 사진을 더욱 쉽게 찾을 수 있습니다. 모든 사진을 일일이 확인해 찾는 방식이 아니라 "해변", "파티", 같은 카테고리로 사진을 찾을 수 있습니다.
자동 카테고리 API를 경험해 보시고 싶으시면 아래 링크로 접속 해 주세요
커스텀 트레이닝
커스텀 트레이닝을 통해 고객들은 자신의 필요에 맞게 키워드 혹은 카테고리를 정할 수 있습니다. 커스텀 트레이닝을 통해 고객들은 더 높은 정확도와 자신들만의 특별한 키워드를 지정 할 수 있습니다.
커스텀 트레이닝의 방법은 아주 간단합니다. 고객님이 카테고리와 그 카테고리에 맞는 샘플 데이터를 제공해 줍니다. 그 데이터를 이용해 시스템을 교육 시킵니다. 새롭게 생성된 분류자는 API에 쉽게 적용이 되 기존에 있는 시스템과 작업에 간단히 설치 할 수 있습니다. 이 방법은 사진에 주석을 달거나 분류를 수동으로 하고 있었던 기업에게 아주 적합합니다. 기존 분류 된 컨텐츠는 분류되지 않은 나머지 컨텐츠에 대한 학습 데이터 세트로 사용될 수 있습니다.
커스텀 트레이닝에 대해 더 자세히 알고 싶으시면 아래 링크로 접속 해 주세요
온프레미스 솔루션
저희가 이미지 인식 회사로서 최초로 발표한 온프레미스(사내 서버 직접 설치) 솔루션은 클라우드 버전과 동일한 신뢰성과 결과를 줍니다.
고객의 하드웨어와 OS가 최소 조건을 충족시키면 설치가 가능 합니다. 처리 할 수 있는 사진 개수와 처리 속도는 개인 서버 사이즈에 의해 결정 됩니다. 처음 설치를 할 때 비용과 시간이 더 소요 되지만, 직접 개발 하는 비용에 비해 훨씬 더 효율적입니다.
아직도 저희가 어떻게 도움이 될지 확신이 안 서십니까? 저희 대표 사례를 보시고 결정하세요.
기아 자동차
기아 자동차는 K5차량의 쇼케이스 캠페인을 진행했습니다. 개인의 라이프 스타일에 맞는 K5 모델을 자동으로 추천하는 형식으로 진행했습니다. 36가지의 다른 라이프 스타일에 맞는 옵션을 제공하였습니다. 차 엔진과, 색상 그리고 다른 디자인에 의해 추천을 완료 했습니다.
Imagga의 기술은 이 캠페인에서 개인의 SNS 사진을 분석하는 코어 기술을 제공했습니다. SEEDPOST와 협력을 통해 캠페인에 적합한 프로파일링 엔진을 만들었습니다. Imagga의 자동 태깅 API를 이용해 SEEDPOST는 이용자의 성향과 일치하는 K5모델을 자동으로 추천하는 엔진을 성공적으로 만들었습니다.
사용자 프로파일링에 어떻게 기술이 적용되었는지 궁금하시면 연락 주시기 바랍니다.
스위스콤
Imagga의 API는 스위스콤의 개인 클라우드 저장소에 온프레미스 솔루션으로 설치되었습니다. Imagga의 기술은 클라우드 저장소에 사진 검색 및 찾아보기 기능을 추가하는데 사용되었습니다. 저희의 기술은 실 이용자들에게 사진 검색과 찾아보기 기능을 제공하며, 이 기술로 인해 실제 프로그램을 사용하는 시간과 이용자를 증가 시켰습니다.
이마가의 온프레미스 솔루션에 대해 더 정보를 얻고 싶으신가요? 연락 주십시오.
Image Detection Bots and Imagga
Looks like 2016 will be the year of bots and almost all of them pretend to use some form of machine learning to do their job even better. Microsoft just released an image detection bot called CaptionBot. As all related to image recognition via machine learning, the bot does quite a good job detecting some concepts and quite a bad when it comes to others.
We always find it fun to compare image recognition services and see how Imagga performs with photos used by users. This blog post was provoked by a friend who posted this on Facebook:
It’s about an article on TNW about Microsoft CaptionBot written by Natt Garun and it’s failure to deliver consistent results. We love Microsoft and even use Azure and Microsoft's Translation tools to improve the language support for tags, but it’s quite tempting to run the test and share.
We’ve run the same photos included in the article through Imagga API demo, and that’s what our image recognition thinks about it. It’s still not perfect! Have fun exploring the results.
The first photo is of a home metal hook:
We've nailed that one, seriously!
Obvious description of the photo will be something like Beautiful Asian girl siting in a vegie garden. We are quite close to it with tags like yard, grass, outdors, people. Well, we didn’t figure the rase properly, but that’s something we are cureenlty working on and lots of imporvements coming in couple of months.
There’s something about that photo that makes it difficult for the tech to figure out. We didn’t go exeptionally good job here. Well, at least we tagged with happy, witch is obvious from the shaning face of that Asian woman.
It’s obvious we do well with living creatures. Thanks good we do not need to figure out if they are male or female and how happy they are.
This one is quite dificult as it brings alot of contect, that the machine need to know about. We haven’t trained for movie characters, but tags are in general good. Well, we can argue about the guitar - if you have wild imagination you can definately picture Darth Vader with guitar. Human do not apply much in this case, and we are happy with the low confidence level ;-)
Ever wondered where do people come up with ideas for testing? This one is classic. Definately no surfboard and yes, it’s an attractive fashionable male model ;-)
Toy elephents may look like that in the forseeble future, but to us this looks like a man robot. Automation is all what robots are predestened for, right?
Can’t agree more that the person laying in the grass looks like an alligator. Tags like summer and grass bring a bit of balance to the results. If not accurate at least a bit closer then a cow laying down in a grassy field.
This one is quite though. What’s quite visible for a person to see, it’s not so visible for the machine, who’s lacking the context of the two mager objects on this photos.
It’s great idea to put bed next to any desk in the office. May be one next to the copy machine will not be that good idea - to much noice and traffic arround it.
We’ve tried to describe this picture and results are not too bad ;-)
Last but not least, we do not see donuts but happy, male cople in love. Well, we’ve got a bit of spicy results with less confidence as wife and caucasian, but still…