SBIR-STTR Award

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project is to develop and commercialize visual search for fine-grained recognition of products and style in interior decor and apparel. The technology will help the broader public find items that may be difficult to search for using traditional text-based search. In many markets (home decor, fashion, etc.), customers seek products that have unique visual appearances that cannot easily be expressed with a text-based search. This project will develop visual search tools for the home decor and apparel markets. This Small Business Innovation Research (SBIR) Phase I project will develop software based on deep learning for product and apparel recognition and style recognition. Our prior prototype uses deep learning to recognize specific products from "regular" photographs taken by customers, where the challenge is that these regular photos of products can have many different backgrounds, sizes, orientations, or lighting when compared to the iconic product image, and the product could be significantly occluded by clutter in the scene. The goal of this project is to generalize the work to achieve broad applicability through four major objectives: Generalizing the settings and product categorization and taxonomy to support a broad range of customers and product types (Objective 1); semi-automatic detection of products in scene images to scale to large photo collections (Objective 2); refining the trained models for fine-grained matches to meet customer needs (Objective 3); and deploying the system live to companies (Objective 4).

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project is to develop visual search for product recognition in the furniture and home décor vertical. Text-based searches have revolutionized the ability of people to complete tasks more quickly and efficiently as they are able to find the information they desire in an organized, compiled, and logical manner. Visual search provides the next level of disruption in search capabilities by allowing users to find information even more rapidly and accurately by using images. The deep learning-based software being developed will allow consumers to find products they are interested in, and co-purchase related products, quickly. Further, users will be more engaged through exposure to designer photographs of products (inspirational photography). By helping customers find exactly what they are looking for in a timely manner, user engagement and productivity will be increased. Further, related style-based recommendations will increase purchasing overall. Increased spending stimulates economic growth by increasing taxable revenue by retailers, and through increased sales taxes generated from the purchases. This Small Business Innovative Research Phase II project seeks to develop a visual search engine that is poised to disrupt retail and ecommerce by switching the focus from text-based to visual search-based exploration. The platform initially targets interior décor and furniture where deep learning techniques are trained to recognize products across a wide range of conditions. In Phase II, the software deep learning architectures will be generalized to enable a broader range of products, and to allow customers more control over design decisions and choices. A client-facing REST API will allow retailers, designers, and media companies to programmatically access functionality of the platform, and build their own user interfaces and apps on top of the deep learning technology. Lastly, it is proposed to develop a white-label app that can be customized for individual retailers who want to distribute this visual search capability to their customers. Achieving these objectives will create state-of-the-art performance in visual search for applications in interior design, apparel search, real estate search, and product look-up.