It looks like students who are a member of another group can participate in chatroom of other groups in a Group Mode Visible Group.

However, the description from https://docs.moodle.org/39/en/Groups#Visible_groups says that:

• No groups - There are no sub groups, everyone is part of one big community
• Separate groups - Each group can only see their own group, others are invisible.
• Visible groups - Each group works in their own group, but can also see other groups. (The other groups' work is read-only.)

Steps to replicate:

1. Create a course and enrol two student users.
2. Create a group (groupA), member: studentA
3. Create another one (groupB), member: studentB
4. Turn editing on.
5. Add a Chat activity and set Common module settings / Group mode = Visible groups.
6. Click the "Save and return to course" button.
7. Log in as studentA, go to the course contents page and visit the Chat activity.
8. Remember that studentA is a member of groupA. In the Visible groups dropdown control, select Other groups / Group B.
9. Click on the link "Use more accessible interface" and that will open up a new window.
10. Type in a message and click the "Submit" button, then close the window.
11. Log back as an admin/teacher and visit the Chat activity.
12. On the Visible groups dropdown control, select "groupB".
13. Click on "View past chat sessions" link then click the "List all sessions" link.
    RESULT:
    You will see that the message that was submitted by studentA of groupA was posted in the session of groupB.