regular_expressions

= Python - Regular Expressions =

  



A //regular expression// is a special sequence of characters that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world. The module **re** provides full support for Perl-like regular expressions in Python. The re module raises the exception re.error if an error occurs while compiling or using a regular expression. We would cover two important functions which would be used to handle regular expressions. But a small thing first: There are various characters which would have special meaning when they are used in regular expression. To avoid any confusion while dealing with regular expressions we would use Raw Strings as **r'expression'**. = The //match// Function = This function attempts to match RE //pattern// to //string// with optional //flags//. Here is the syntax for this function: code re.match(pattern, string, flags=0) code || Here is the description of the parameters: The //re.match// function returns a **match** object on success, **None** on failure. We would use//group(num)// or //groups// function of **match** object to get matched expression.
 * ~ Parameter ||~ Description ||
 * pattern || This is the regular expression to be matched. ||
 * string || This is the string which would be searched to match the pattern ||
 * flags || You can specifiy different flags using exclusive OR (|). These are modifiers which are listed in the table below. ||
 * ~ Match Object Methods ||~ Description ||
 * group(num=0) || This methods returns entire match (or specific subgroup num) ||
 * groups || This method return all matching subgroups in a tuple (empty if there weren't any) ||

Example:
code #!/usr/bin/python import re

line = "Cats are smarter than dogs";

matchObj = re.match( r'(.*) are(\.*)', line, re.M|re.I)

if matchObj: print "matchObj.group : ", matchObj.group print "matchObj.group(1) : ", matchObj.group(1) print "matchObj.group(2) : ", matchObj.group(2) else: print "No match!!" code || This will produce following result: code matchObj.group: Cats are matchObj.group(1) : Cats matchObj.group(2) : code || = The //search// Function = This function search for first occurrence of RE //pattern// within //string// with optional //flags//. <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Here is the syntax for this function: code <span style="font-family: 'Courier New',monospace; font-size: 12px;">re.search(pattern, string, flags=0) code || <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Here is the description of the parameters: <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">The //re.search// function returns a **match** object on success, **None** on failure. We would use//group(num)// or //groups// function of **match** object to get matched expression.
 * ~ Parameter ||~ Description ||
 * pattern || This is the regular expression to be matched. ||
 * string || This is the string which would be searched to match the pattern ||
 * flags || You can specifiy different flags using exclusive OR (|). These are modifiers which are listed in the table below. ||
 * ~ Match Object Methods ||~ Description ||
 * group(num=0) || This methods returns entire match (or specific subgroup num) ||
 * groups || This method return all matching subgroups in a tuple (empty if there weren't any) ||

Example:
code <span style="font-family: 'Courier New',monospace; font-size: 12px;">#!/usr/bin/python import re

line = "Cats are smarter than dogs";

matchObj = re.search( r'(.*) are(\.*)', line, re.M|re.I)

if matchObj: print "matchObj.group : ", matchObj.group print "matchObj.group(1) : ", matchObj.group(1) print "matchObj.group(2) : ", matchObj.group(2) else: print "No match!!" code || <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This will produce following result: code <span style="font-family: 'Courier New',monospace; font-size: 12px;">matchObj.group: Cats are matchObj.group(1) : Cats matchObj.group(2) : code || = Matching vs Searching: = <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Python offers two different primitive operations based on regular expressions: **match** checks for a match only at the beginning of the string, while **search** checks for a match anywhere in the string (this is what Perl does by default).

Example:
code <span style="font-family: 'Courier New',monospace; font-size: 12px;">#!/usr/bin/python import re

line = "Cats are smarter than dogs";

matchObj = re.match( r'dogs', line, re.M|re.I) if matchObj: print "match --> matchObj.group : ", matchObj.group else: print "No match!!"

matchObj = re.search( r'dogs', line, re.M|re.I) if matchObj: print "search --> matchObj.group : ", matchObj.group else: print "No match!!" code || <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This will produce following result: code <span style="font-family: 'Courier New',monospace; font-size: 12px;">No match!! search --> matchObj.group : dogs code || = Search and Replace: = <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Some of the most important **re** methods that use regular expressions is **sub**.

Syntax:
code <span style="font-family: 'Courier New',monospace; font-size: 12px;">re.sub(pattern, repl, string, max=0) code || <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This method replace all occurrences of the RE //pattern// in //string// with //repl//, substituting all occurrences unless //max// provided. This method would return modified string.

Example:
<span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Following is the example: code <span style="font-family: 'Courier New',monospace; font-size: 12px;">#!/usr/bin/python

phone = "2004-959-559 #This is Phone Number"

num = re.sub(r'#.*$', "", phone) print "Phone Num : ", num
 * 1) Delete Python-style comments

num = re.sub(r'\D', "", phone) print "Phone Num : ", num code || <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This will produce following result: code <span style="font-family: 'Courier New',monospace; font-size: 12px;">Phone Num : 2004-959-559 Phone Num : 2004959559 code || = Regular-expression Modifiers - Option Flags = <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Regular expression literals may include an optional modifier to control various aspects of matching. The modifier are specified as an optional flag. You can provide multiple modified using exclusive OR (|), as shown previously and may be represented by one of these: = Regular-expression patterns: = <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Except for control characters, **(+ ? . * ^ $ [ ] { } | \)**, all characters match themselves. You can escape a control character by preceding it with a backslash. <span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">Following table lists the regular expression syntax that is available in Python. = Regular-expression Examples: =
 * 1) Remove anything other than digits
 * ~ Modifier ||~ Description ||
 * re.I || Performs case-insensitive matching. ||
 * re.L || Interprets words according to the current locale.This interpretation affects the alphabetic group (\w and \W), as well as word boundary behavior (\b and \B). ||
 * re.M || Makes $ match the end of a line (not just the end of the string) and makes ^ match the start of any line (not just the start of the string). ||
 * re.S || Makes a period (dot) match any character, including a newline. ||
 * re.U || Interprets letters according to the Unicode character set. This flag affects the behavior of \w, \W, \b, \B. ||
 * re.X || Permits "cuter" regular expression syntax. It ignores whitespace (except inside a set [] or when escaped by a backslash), and treats unescaped # as a comment marker. ||
 * ~ Pattern ||~ Description ||
 * ^ || Matches beginning of line. ||
 * $ || Matches end of line. ||
 * . || Matches any single character except newline. Using m option allows it to match newline as well. ||
 * [...] || Matches any single character in brackets. ||
 * [^...] || Matches any single character not in brackets ||
 * re* || Matches 0 or more occurrences of preceding expression. ||
 * re+ || Matches 1 or more occurrence of preceding expression. ||
 * re? || Matches 0 or 1 occurrence of preceding expression. ||
 * re{ n} || Matches exactly n number of occurrences of preceding expression. ||
 * re{ n,} || Matches n or more occurrences of preceding expression. ||
 * re{ n, m} || Matches at least n and at most m occurrences of preceding expression. ||
 * a| b || Matches either a or b. ||
 * (re) || Groups regular expressions and remembers matched text. ||
 * (?imx) || Temporarily toggles on i, m, or x options within a regular expression. If in parentheses, only that area is affected. ||
 * (?-imx) || Temporarily toggles off i, m, or x options within a regular expression. If in parentheses, only that area is affected. ||
 * (?: re) || Groups regular expressions without remembering matched text. ||
 * (?imx: re) || Temporarily toggles on i, m, or x options within parentheses. ||
 * (?-imx: re) || Temporarily toggles off i, m, or x options within parentheses. ||
 * (?#...) || Comment. ||
 * (?= re) || Specifies position using a pattern. Doesn't have a range. ||
 * (?! re) || Specifies position using pattern negation. Doesn't have a range. ||
 * (?> re) || Matches independent pattern without backtracking. ||
 * \w || Matches word characters. ||
 * \W || Matches nonword characters. ||
 * \s || Matches whitespace. Equivalent to [\t\n\r\f]. ||
 * \S || Matches nonwhitespace. ||
 * \d || Matches digits. Equivalent to [0-9]. ||
 * \D || Matches nondigits. ||
 * \A || Matches beginning of string. ||
 * \Z || Matches end of string. If a newline exists, it matches just before newline. ||
 * \z || Matches end of string. ||
 * \G || Matches point where last match finished. ||
 * \b || Matches word boundaries when outside brackets. Matches backspace (0x08) when inside brackets. ||
 * \B || Matches nonword boundaries. ||
 * \n, \t, etc. || Matches newlines, carriage returns, tabs, etc. ||
 * \1...\9 || Matches nth grouped subexpression. ||
 * \10 || Matches nth grouped subexpression if it matched already. Otherwise refers to the octal representation of a character code. ||

Literal characters:

 * ~ Example ||~ Description ||
 * python || Match "python". ||

Character classes:

 * ~ Example ||~ Description ||
 * [Pp]ython || Match "Python" or "python" ||
 * rub[ye] || Match "ruby" or "rube" ||
 * [aeiou] || Match any one lowercase vowel ||
 * [0-9] || Match any digit; same as [0123456789] ||
 * [a-z] || Match any lowercase ASCII letter ||
 * [A-Z] || Match any uppercase ASCII letter ||
 * [a-zA-Z0-9] || Match any of the above ||
 * [^aeiou] || Match anything other than a lowercase vowel ||
 * [^0-9] || Match anything other than a digit ||

Special Character Classes:

 * ~ Example ||~ Description ||
 * . || Match any character except newline ||
 * \d || Match a digit: [0-9] ||
 * \D || Match a nondigit: [^0-9] ||
 * \s || Match a whitespace character: [ \t\r\n\f] ||
 * \S || Match nonwhitespace: [^ \t\r\n\f] ||
 * \w || Match a single word character: [A-Za-z0-9_] ||
 * \W || Match a nonword character: [^A-Za-z0-9_] ||

Repetition Cases:

 * ~ Example ||~ Description ||
 * ruby? || Match "rub" or "ruby": the y is optional ||
 * ruby* || Match "rub" plus 0 or more ys ||
 * ruby+ || Match "rub" plus 1 or more ys ||
 * \d{3} || Match exactly 3 digits ||
 * \d{3,} || Match 3 or more digits ||
 * \d{3,5} || Match 3, 4, or 5 digits ||

Nongreedy repetition:
<span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This matches the smallest number of repetitions:
 * ~ Example ||~ Description ||
 * <.*> || Greedy repetition: matches " perl>" ||
 * <.*?> || Nongreedy: matches " " in " perl>" ||

Grouping with parentheses:

 * ~ Example ||~ Description ||
 * \D\d+ || No group: + repeats \d ||
 * (\D\d)+ || Grouped: + repeats \D\d pair ||
 * ([Pp]ython?)+ || Match "Python", "Python, python, python", etc. ||

Backreferences:
<span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This matches a previously matched group again:
 * ~ Example ||~ Description ||
 * ([Pp])ython&\1ails || Match python&rails or Python&Rails ||
 * (['"])[^\1]*\1 || Single or double-quoted string. \1 matches whatever the 1st group matched . \2 matches whatever the 2nd group matched, etc. ||

Alternatives:

 * ~ Example ||~ Description ||
 * python|perl || Match "python" or "perl" ||
 * rub(y|le)) || Match "ruby" or "ruble" ||
 * Python(!+|\?) || "Python" followed by one or more ! or one ? ||

Anchors:
<span style="background-color: #ffffff; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: justify;">This need to specify match position
 * ~ Example ||~ Description ||
 * ^Python || Match "Python" at the start of a string or internal line ||
 * Python$ || Match "Python" at the end of a string or line ||
 * \APython || Match "Python" at the start of a string ||
 * Python\Z || Match "Python" at the end of a string ||
 * \bPython\b || Match "Python" at a word boundary ||
 * \brub\B || \B is nonword boundary: match "rub" in "rube" and "ruby" but not alone ||
 * Python(?=!) || Match "Python", if followed by an exclamation point ||
 * Python(?!!) || Match "Python", if not followed by an exclamation point ||

Special syntax with parentheses:

 * ~ Example ||~ Description ||
 * R(?#comment) || Matches "R". All the rest is a comment ||
 * R(?i)uby || Case-insensitive while matching "uby" ||
 * R(?i:uby) || Same as above ||
 * rub(?:y|le)) || Group only without creating \1 backreference ||

<span style="background-color: #ffffff; color: #900b09; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: left;"> <span style="background-color: #ffffff; color: #900b09; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: left;"> <span style="background-color: #ffffff; color: #900b09; font-family: verdana,helvetica,arial,sans-serif; font-size: 11px; text-align: left;">