It appears that changing the aggregation while the course total is set to "hide" makes grade items activity hidden to.

REPLICATION:

1. Create a course.
2. Turn edit mode to On.
3. Add an assignment activity.
4. Go to gradebook setup page.
5. RESULT: grade items (all visible). Aggregation = Natural
   1. assign1
   2. course/site total

7. Edit the course/site total and set to "Hidden".

8. Change Aggregation from Natural to "Mean of grades" and save.

9. RESULT: The activity (assignment1) is now hidden too.

Tested this on courses at https://qa.moodledemo.net/