Density-Refine: Patent Image Retrieval by Density-Based Region Extraction and Feature Fusion

Design-by-analogy (DbA) is an important methodology in mechanical design that generates innovative solutions in the target domain with inspiration from a source domain. The patent database is one of the valuable source domains for the DbA method. Meanwhile, patents are crucial in engineering, especially for engineering design and acquiring an exclusive business advantage. Therefore, efficient patent exploration is essential in patent application and design inspiration. Patent image complements text-based descriptions with visual information. The visual information is practical for patent devices with complex structures. We found that spatial density is vital in extracting the relevant subregions. Therefore, we leveraged this property by incorporating density-based clustering to enrich the training dataset. We also proposed a feature fusion mechanism to utilize the newly extracted subregion information. As a result, we named our method Density-Refine since we improved the performance of patent image retrieval by employing the density property. Our method outperformed the state-of-the-art approaches in the benchmark dataset for patent image retrieval. We also investigate the performance of applying the density property to other similar mediums, such as sketch image retrieval. We expect this work to be a stepping stone to inspire more influential studies in image retrieval and design inspiration.