There is provided with an image processing apparatus for performing class classification of an image. A generation unit generates, for each class, a feature of an image group belonging to the class based on classification information for classifying each of a plurality of images to one of a plurality of classes in accordance with an object contained in an image. A determination unit determines, using the feature of each class, a possibility that the image group belonging to the class includes an image containing another object different from the object corresponding to the class.