Laravel Issue with Policy not being triggered properly?

So I have a route model binding like that:

Route::model('questionnaireAttachment', \App\Models\QuestionnaireAttachment::class);

I also have policy registered in AuthServiceProvider.php:

QuestionnaireAttachment::class => QuestionnaireAttachmentPolicy::class,

and the route I have is like this:

Route::get('questionnaire/attachments/{questionnaireAttachment}', [
    AttachmentController::class,
    'getAttachments'
])
->name('questionnaireAttachmentRetrieve')
->middleware('can:view,questionnaireAttachment');

the policy has return false; in it so I know it fails however when I make the request to the endpoint, despite the return false; i still get to the controller and I get a valid 200 response

 public function getAttachments(QuestionnaireAttachment $attachment): \Symfony\Component\HttpFoundation\StreamedResponse
        {
            $file = Storage::disk('local')->exists("{$attachment->path}/{$attachment->filename}");
            if (!$file) {
                return Storage::disk('s3')->download("{$attachment->path}/{$attachment->filename}", $attachment->filename);
            }
            return Storage::disk('local')->download("{$attachment->path}/{$attachment->filename}", $attachment->filename);
        }

via Chebli Mohamed