I'm importing XML file values to DB and end up checking if the row already exists, this takes a lot of time as the table has around 10million rows. For a file to import it takes roughly 90 seconds, on empty database 4secs. I'm checking if the row exists with laravel eloquent exists() method. I want to reduce this time as much as possible, if im importing 10k files, it takes DAYS. I was thinking of saving inserted rows in redis cache and checking if row exists from cache, but if for some reason cache is cleared I don't want to make duplicates in database. Is there anything better than redis?
via Chebli Mohamed
Aucun commentaire:
Enregistrer un commentaire