Remove relationship between Eloquent records

I've created two classes extending Eloquent (contacts and tags), they have a ManyToMany relationship. I'm trying to create the method for un-tagging a contact, but am unable to find any documentation to tell how to remove the entry in the relation-table without deleting either the tag itself or the contact.

So far, I've tried

$contact = Contact::find($contact_id);
$tag = $contact->tags->where('id', '=', $id);
$tag->delete();

This only deletes the contact. It makes sense that it doesn't work, but I'm not sure what else to try. I don't want to delete the contact or the tag, just the relationship between the two.

via Chebli Mohamed