How would I be able to get the foreign key name in another table using nested eager loading with eloquent and laravel

So I currently have 3 models Specie Type User. I want to be able to get the name of the last modified user in relation to the Specie model

The relationships are as follows

Class Specie extends Model {
   public function type()
   {
        return $this->belongsTo('App\Type', 'type_id');
   }

   public function user()
   {
        return $this->belongsTo('App\User', 'user_id');
   }
}

Class Type extends Model {
   public function specie()
   {
        return $this->hasMany('App\Specie', 'type_id');
   }
}

Class User extends Model {
   public function specie()
   {
        return $this->hasMany('App\Specie', 'user_id');
   }
}

I have tried this

Class Specie extends Model {
    public function lastModified()
    {
        return $this->belongsTo('App\User', 'last_modified_by');
    }
}

And then used the below code

$this->type->with(['specie.lastModified' => function ($query) use 
        ($userId) {
            $query->where('user_id', $userId);
        }])->get();

Unfortunately this does not seem to work

However, by using this code

$this->type->with(['specie' => function ($query) use ($userId) {
            $query->where('user_id', $userId);
        }])->get();

I am able to get this:

"id": 1,
"type": "Halo",
"created_at": "2019-07-20 13:02:53",
"updated_at": "2019-07-20 13:02:53",
"specie": [
  {
    "id": 6,
    "user_id": 1,
    "type_id": 1,
    "note": "et",
    "last_modified_by": 1,
  },
  {
    "id": 7,
    "user_id": 1,
    "type_id": 2,
    "note": "sa",
    "last_modified_by": 2,
  },
]

However, what I want to get is the name of the last modified person name which is the primary key in the User model and the foreign key in the Specie model

This is what I expect to get:

"id": 1,
"type": "Halo",
"created_at": "2019-07-20 13:02:53",
"updated_at": "2019-07-20 13:02:53",
"specie": [
  {
    "id": 6,
    "user_id": 1,
    "type_id": 1,
    "note": "et",
    "last_modified_by": 1,
    "user": [
        {
         "id": 1,
         "user": 'gerrard'
        }
    ]
  },
  {
    "id": 7,
    "user_id": 1,
    "type_id": 2,
    "note": "sa",
    "last_modified_by": 2,
    "user": [
         {
           "id": 2,
           "user": 'chris'
         }
    ]
  }
]

via Chebli Mohamed