This website scraper was developed by Kevin Rio, a Miami Website Developer,  for the purpose of collecting information and content from affiliate sites, such as E-Commerce applications that do not provide an API or an externally available database. It is also useful for SEO professionals to evaluate competitor websites.

This is a completely free website scraper that can be used in any commercial or private applications.

What This Website Scraper Does

This web scraper collects a website address as input from the user and retrieves all of the HTML code from that site. It then filters all of the HTML and JavaScript code using regular expressions and leaves only the website’s text for the user to explore. It provides the text in an easy to copy text area and a variable that is easy to manipulate and extend in your own scripts.

The download provides all of the files including the Visual Studio solution file.

What It Does

My Links Central makes it easy to manage your back-links on your content sites.

• Manage sidebars of multiple WordPress sites from one control panel
• Easily add links to one central file that all widgets will feed from
• Specifically made so that SEO experts can easily create links for all of their existing blogs
• Add both a URL and the Anchor tag text. The title tag is automatically taken from the anchor text
• No extra database needed. Everything is XML driven
• Easily style the links using the class MyLinksCentral as an unordered list

Usage

There are two separate zip files. The first is the widget, which you install into WordPress. The second is the administrator panel that you upload via FTP to your main web server.

Configuring the Admin Panel

• Upload the administrator files to a folder via FTP to whatever server you want
• Navigate to the LinksControl.php file via your web browser
• Click the ‘Add’ button and input the URL and Title of the links you want to add

Configuring the Widget

• Install the widget via the plugins admin panel and go to the widgets page and drag it over
• In the XML file box, put the absolute file path to the XML file that is located with the admin panel

Download

Download the Admin Panel

Download the Widget

I had to develop a web scraper for a proof-of-concept application a few months ago in PHP and since have had a few questions on how to implement them. I decided to make a post and include the web scraper free of charge for other developers to download and utilize in their scripts.

What This Scraper Does

This scraper uses the cURL PHP library in combination with 10 of the most popular search engines to gather information. First, it takes a string as a query provided by the user and searches the internet for sites relevant to that search query. It then visits and scrapes these sites for whatever regular expression you provide to it and stores these results into an array that can be placed into a database, xml file, or anything similar.

I developed the entire code base from scratch for the Olympus Tours E-Commerce website. Olympus Tours competes heavily on Google for the top spot on many keywords. They wanted to ensure that the code was clean and the shopping cart was fully customized to meet their needs. I developed a full back-end control panel so that they could control products, prices, keywords, and content. I also developed a light-weight, fully customizable shopping cart. With hundreds of available tours and transportation to hundreds, if not thousands of hotels, this was big undertaking. I also sliced the design created by their in-house graphic designer and implemented it into the CMS and shopping cart I developed. The site’s code is fully semantic and SEO friendly.

I developed the code base for the JetFlicks! iPhone and Android application, which allows users to watch TV episodes directly from their cell phones. It is a user-subscription model where users choose a subscription plan and are billed monthly or annually for their subscription. As of February 2010, this application was rated #1 on the Android marketplace. Search for JetFlicks! on your android or iPhone device (iPhone coming March 2010).

Since the launch of their website, the Bandits have been featured on Miami Dolphins TV and other multimedia outlets.

Miss-Ink asked Krio Media to develop a functional, yet easy to navigate site with social media features, such as a live twitter feed. This site is heavily based upon HCI principles and getting users attention to the content. This WordPress enabled blog is simple to update and manage.