Filter product with attributes in Laravel

I have these two tables:

Product Table :

 |id  | Title     | Price|
 -------------------------
 |1   | Title 1   |  5000|
 |2   | Product 2 |  7000|

and the other is

product_attribute table:

 |id | product_id | attribute_id | attribute_name | value|
 ---------------------------------------------------------
 |1    | 1        |       5      |     Color      |  Red |
 |2    | 1        |       6      |     Size       |  XL  |
 |3    | 2        |       5      |     Color      | Green|

Product and Product attribute is related with following relation (In the product model):

 public function attributes()
    {
        return $this->hasMany(ProductsAttribute::class, 'product_id ');
    }

I'm fetching data like this :

return Product::with('attributes')
        ->whereHas('attributes', function ($query) use ($attribute_id,$attribute_value){
       if (!empty($attribute_id) && !empty($attribute_value)) {
            $query->whereIn('attribute_id', $attribute_id);
            $query->whereIn('value', $attribute_value);
        }
        })
        ->paginate(10);

There are multiple attributes of a product, in present case even a single match to a given attribute value & attribute id returns the product irrespective of other attribute id and value matches or not but what i want is if any of the given product attributes doesn't match (i.e. its attribute_id and attribute_value is not in the product attribute table then that product should not return in the result).

eg. If user has selected color with attribute id 5 and value red and size of attribute id 6 and value xl then only Product with title 1 should be returned. if user hase selected attribute_id 5 and and value red but attribute size does'nt match then then product should not return.

Any help is highly appreciated.

via Chebli Mohamed