algorithm to auto-download articles from the internet


I have an issued in my homeworks and I thinked if there is an rxisted algorithm or if can i create new one that takes key words like "germany" and "polution" and parses in google scholar. It parses fpr example the 10 first reults and each time it finds the key words in a specific part from the article ( just in the introduction) it downloads it.

Any one can help me ith any infrmation that can help me in this idea? Is it possible? what kind of alforithms should it be? ...


Posted 2018-12-04T19:46:47.687

Reputation: 9



what you are asking for is known as web scraping , if you don't what it is, you can read about it. can you please clarify your question more, so you can get the right answer.

what programming languages you'll be using, if you are planning on using python, there's a great framework for web scraping called Scrapy. I found this Parser which does what you are asking for. finally, you can check this related question.

Reslan Tinawi

Posted 2018-12-04T19:46:47.687

Reputation: 56

Oh yeah!! thank you this is what i'am searshing exactly. I didn't choose the language yet I'm searshing for the most appropriate for this aim. I want actually to automatically download from google schoolar all the articles that contains in the abstract part the words "germany" and "polution" – Linda – 2018-12-09T18:33:39.043