Perl LWP UserAgent Example: Get a http or ftp data

Perl LWP UserAgent is an amazing tool for automating web queries. Here is the simple text to get HTTP or FTP data using the Perl LWP UserAgent:

#!/usr/bin/perl
# pretty standard to have this at least in all scripts
# at least for development.  Use taint also for CGI.
use warnings;
use strict;
use LWP::UserAgent;
use CGI qw(header -no_debug);
# if we don't do this it will buffer and then print results
# I want to see results immediately as it happens.
# not every server configuration supports this
$|++;
# put a page that requires authentication here to see a 401 error
# put a normal URL here to see it pull text normally.
my $URL = 'http://digitalcrunch.com/';
# create a new object and request the GET method on $URL
my $res = LWP::UserAgent->new->request(new HTTP::Request GET => $URL);
# required for any type of CGI
print header;
# at this point you can either use the print statement,
# or assign the data and manipulate it
# you can can also just print the text:
# print $res->is_success ? $res->cotnent : $res->status_line;
# but lets assume we want to manipulate further before printing:
my $data = $res->is_success ? $res->content : $res->status_line;
# simple regex to grab a pattern match from the page:
if ($data =~ m/The quick brown(.*)jumped over the lazy dog/) {
  print "we found a ' $1 ' (should be ' fox ')";
} else {
  print "probably not a typing tutorial!\n";
}
# What did the full data contain?
print $data;
# not required, but a good habit.
exit;