Simple Python HTML,XML DOM Parser

This parser is intended to be simple and easy to use. You can download a copy from github . The parser can detect the various forms of tags like id=lorem , id='lorem', id="lorem". I have been using this script for my current scraping project.


I had been using the python module minidom but it's no use for the malformed xml and html . So I wrote this script using only regular expression.

Usage:

Copy the spp.py in your working directory and import the package.

 Instantiate the parser class by providing the url  or xml|html string

 To Get the 'id' attribute of the 'style' tag
 1.spp.parser('http://www.google.com').getByTag('style').item(0).attr('id')

 To get the attribute of a node whose id is 'csi'

 2.spp.parser('http://www.google.com').getById('csi).attr('style')

 # doc is the html or xml string you want to parse

 3.spp.parser(doc).getById('test').attr('href')

 To get the src of an image which is the 4th image of the document
 4.spp.parser(doc).getBytag('img').item(4).attr('src')

 To get the content of a node which has no child
 5.spp.parser(doc).getById('test').innerText()

There are some nice parsing tools for python available.

1. Beautiful Soup
2. lxml

PHP regex lookaround assertion: lookbehind and lookahead

Regular Expression is a mighty beast, if you can tame it Regex will be a great leverage to solve your Problems.

A Real World Example:

Regex assertion is to check if any string matches following or the preceding characters. Suppose we want to check if there is any '.mp3'or '.mp4' string in our text but we only want the value 'mp3'or 'mp4'

One solution is:

$text = 'demo.mp3, shakira.mp4,lorum ipsum';

if(preg_match_all('/\.mp3|\.mp4/', $text,$matches)):

foreach($matches as $key => $match){
$matches[$key] =  str_replace('.', '' ,$matches[$key]);
}

endif;
var_dump($matches);

But this is ugly, we can use the lookbehind assertion to make the code simple:

preg_match_all('/(?<=.)mp3|mp4/', $text,$matches);
var_dump($matches); 

Let's Go Back: Check The Basic: 
 What are the lookahead and lookbehind assertions? and what are the syntax?

Assertions Only check that from the current position if a string matches the following or preceding characters, It does not consume any character. 'lorum(?=ipsum)' matches 'lorum' from the text 'lorumipsum'. Note that it's not matching 'lorumipsum', it's just checking if 'ipsum' are the next characters after 'lorum'.

LookAhead Syntax:
Positive lookahead: (?=)
Negative lookahead: (?!)

LookAhead Assertion Checks if following characters matches(positive lookahead) or does not(negative lookahead) from the current position.

positive Lookahead:
The following example matches 'lorum' first then checks that if 'ipsum' is next, If the checking is successful then function returns a successful match.

preg_match('/lorum(?=ipsum)/','lorumipsum', $match);
var_dump($match); 

Negative Lookahead: The following example matches 'lorum' first then checks that if 'test' is not next, If the checking is successful then function returns a successful match.

preg_match('/lorum(?!test)/','lorumipsum', $match);
var_dump($match); 

Look Behind Syntax:
Positive lookbehind:(?<=)
Negative lookbehind:(?<!)

positive Lookbehind:
This example matches 'ipsum' first then checks that if 'lorum' is previous, If the checking is successful then function returns a successful match.

preg_match('/(?<=lorum)ipsum/','lorumipsum', $match);
var_dump($match); 

 ******

One important thing to remember: Regex engine searches from left to right but in this case as there is nothing too match before our assertion it matces ipsum first to determine the current position. Suppose we would build our regex expression by  '^(?<=lorum)ipsum' then the regex engine would match the start of the line first. Then it would determine whether 'lorum' matches previous which would obviously fail as there is no character preceding 'lorum'.



Negative LookBehind: 
The following example matches 'ipsum' first then checks that if 'hello' is not previous, If the checking is successful then function returns a successful match.

preg_match('/(?<!hello)ipsum/','lorumipsum', $match);
var_dump($match);