[FIXED] How does java.util.stream.Stream.distinct() method work? Can I override the equals() method of the stream of the objects?

Issue

My use case is that I am trying to use distinct method of Stream to remove Students with same roll number from list of objects of class StudentCourseMapping. Pojo details are below

public class StudentCourseMapping implements Serializable{
    private String name;
    private String dept;
    private Integer roll;
    private String course;

Below is the equals method

    @Override
    public boolean equals(Object obj) {
        StudentCourseMapping other = (StudentCourseMapping) obj;
        if (roll == null) {
            if (other.roll != null)
                return false;
        } else if (!roll.equals(other.roll))
            return false;
        return true;
    }

Below is the implementation

public class RemoveDuplicateUsingStream {
    public static void main(String[] args) {
        List<StudentCourseMapping> studentCourceList = JacksonJSONReaderObjectMapper.jsonReader();
        
        studentCourceList.stream().distinct().forEach(System.out::println);
        StudentCourseMapping s0 = studentCourceList.get(0);
        StudentCourseMapping s1 = studentCourceList.get(1);
        System.out.println(s0.equals(s1));

        Set<Integer> st = new HashSet();
        List<StudentCourseMapping>studentCourceList2 = studentCourceList.stream().filter(s -> st.add(s.getRoll()))
                .collect(Collectors.toCollection(ArrayList::new));
        System.out.println(studentCourceList2.size());
    }
}

And the output is

StudentCourseMapping [name=Alu, dept=Physics, roll=12, course=Quantum Theory]
StudentCourseMapping [name=Alu, dept=Physics, roll=12, course=English]
StudentCourseMapping [name=Sam, dept=Commerce, roll=16, course=English]
StudentCourseMapping [name=Sam, dept=Commerce, roll=16, course=Accounts]
StudentCourseMapping [name=Joe, dept=Arts, roll=19, course=English]
StudentCourseMapping [name=Joe, dept=Arts, roll=19, course=Hindi]
true
3

JacksonJSONReaderObjectMapper.jsonReader(); is a custom method which reads below JSON. I am able to achieve the same by using filter and adding to HashSet but I really want to know what is wrong with my distinct implementation.

{
    "studentCourseMapping": [
        {
            "name": "Alu",
            "dept": "Physics",
            "roll": 12,
            "course": "Quantum Theory"
        },
        {
            "name": "Alu",
            "dept": "Physics",
            "roll": 12,
            "course": "English"
        },
        {
            "name": "Sam",
            "dept": "Commerce",
            "roll": 16,
            "course": "English"
        },
        {
            "name": "Sam",
            "dept": "Commerce",
            "roll": 16,
            "course": "Accounts"
        },
        {
            "name": "Joe",
            "dept": "Arts",
            "roll": 19,
            "course": "English"
        },
        {
            "name": "Joe",
            "dept": "Arts",
            "roll": 19,
            "course": "Hindi"
        }
    ]
}

When I try to test the equals method directly it was working properly and returning true since both the s0 and s1 has roll as 12.

        StudentCourseMapping s0 = studentCourceList.get(0);
        StudentCourseMapping s1 = studentCourceList.get(1);
        System.out.println(s0.equals(s1));

But when I am using distinct all the Objects are getting printed and also while trying to debug in eclipse the distinct method I wrote is not getting called. but the documentation says it should be called. Btw this from Oracle docs 8 but I am using JDK 11

Stream distinct() Returns a stream consisting of the distinct elements (according to Object.equals(Object)) of this stream.

Solution

As @RealSkeptic and @Jesper have already said in the comments, you do have to override the hashCode method within your StudentCourseMapping in order to properly compare the elements in your stream and only keep the distinct ones according to your equals implementation.

Although, this is not mentioned in the Stream documentation nor in the distinct documentation. I believe this is considered implied, as the equals documentation amply covers the obligation in honoring the general hashcode contract when overriding the equals method.

Note that it is generally necessary to override the hashCode method whenever this method is overridden, so as to maintain the general contract for the hashCode method, which states that equal objects must have equal hash codes.

In fact, your snippet produces the desired output once the hashCode() method has been overridden.

public class StudentCourseMapping implements Serializable {
    private String name;
    private String dept;
    private Integer roll;
    private String course;

    //... your class implementation ...

    @Override
    public boolean equals(Object obj) {
        if (obj == null) return false;
        if (obj == this) return true;
        if (obj.getClass() != getClass()) return false;
        StudentCourseMapping student = (StudentCourseMapping) obj;
        return Objects.equals(roll, student.roll);
    }

    @Override
    public int hashCode() {
        return Objects.hashCode(roll);
    }

    @Override
    public String toString() {
        return String.format("[name = %s, dept = %s, roll = %d, course = %s]", name, dept, roll, course);
    }
}

Answered By - Dan
Answer Checked By - Robin (JavaFixing Admin)