Obtaining pivot table data

I have a Users Model and within it I have the following relationship

public function group()
{
    return $this->belongsToMany('App\Group', 'users_user_groups')->withPivot('user_id', 'group_id');
}

I have also set the inverse within the Group model. When complete, I have a users_user_group table with data like so

+------+-------------------+----------+
| id   |           user_id | group_id |
+------+-------------------+----------+
|  755 |                 1 |        1 |
|  756 |                 1 |        2 |
|  757 |                 1 |        3 |
|  758 |                 1 |        4 |
|  759 |                 1 |        5 |
|  760 |                 1 |        6 |
|  761 |                 1 |        7 |
|  762 |                 1 |        8 |
|  763 |                 1 |        9 |
|  764 |                 1 |       10 |
|  765 |                 2 |       11 |
|  766 |                 2 |        7 |
|  767 |                 2 |       10 |
|  768 |                 3 |       12 |
|  769 |                 3 |       13 |

So I know the data is being inserted properly. Now within one of my controllers, I am trying to get all users who are part of the admin group, which has the group_id of 1. So I am doing $users = User::where('active', '=', true)->get();

foreach ($users as $user) {
    if($user->group()->where('groupName', 'admin')) {
        $groupArray[] = $user;
    }
}

For some reason though, every user is added to this array, where only the admins should be added.

I was just looking for advice as to what I am doing wrong? Do I need to link the groupName to the groupId somehow?

Thanks

via Chebli Mohamed