Thank you for visiting this page, this page has been update in another link Store urls into database for later search
If you have millions urls to save for later search, probably you have to fight with database performance simply because urls are saved as text or long char attribute into database, thus, hard to index. Here is a way to let you quickly save, load and search. I save url's md5 checksum to make a index key so later on, use a url's md5 checksum to search it's row in the database. $./idof.pl -r yes -if /home/idofpath/idspathreading input file at 1378507569 ...time elapsed 98 secs for md5 compute 9264153 pnfs mappingtime elapsed 164 secs for loading 9264153 pnfs mappingYou see, the map file before was 2GB, after loaded into db, the db file actually is only 680MB. Save you a lot of disk space, not just speed. $ls -l /home/idofpath/idspath-rw-r--r--. 1 cindy cindy 2108873748 Sep 7 2013 /home/idofpath/idspath$ls -l /home/iddb/idof.db -rw-rw-r-- 1 cindy cindy 687144960 Sep 7 13:52 /home/iddb/idof.dbThe idspath file looks like the map below, url and id separated with '|', the id below has special meaning to me, but for your case, you can put everything you want. To retrive a mapping backhttps://sites.google.com/site/itmyshare/perl-tips-and-examples/store-urls-into-database-for-later-search|000100000000000018B0B918./idof.pl -durl https://sites.google.com/site/itmyshare/perl-tips-and-examples/store-urls-into-database-for-later-search000100000000000018B0B918The whole perl script is attached at bottom, take a try |