ResultScraping - Revision history

Wikiadmin at 21:39, 4 January 2019

2019-01-04T21:39:09Z

Wikiadmin at 21:19, 4 January 2019

2019-01-04T21:19:29Z

Wikiadmin at 15:27, 11 September 2016

2016-09-11T15:27:21Z

Wikiadmin at 08:36, 11 July 2015

2015-07-11T08:36:10Z

Wikiadmin at 07:58, 25 April 2015

2015-04-25T07:58:58Z

Wikiadmin at 16:34, 30 March 2013

2013-03-30T16:34:05Z

Wikiadmin at 13:30, 23 March 2013

2013-03-23T13:30:25Z

Wikiadmin at 19:43, 17 February 2013

2013-02-17T19:43:44Z

Wikiadmin at 17:57, 20 January 2013

2013-01-20T17:57:28Z

Wikiadmin: Created page with "==Results Scraping== Here is a simple Perl script (scrape.pl) which dumps out the open ports: #!/usr/bin/perl use strict; use warnings; use HTML::TableExtract; ..."

2012-04-21T16:39:30Z

Created page with "==Results Scraping== Here is a simple Perl script (scrape.pl) which dumps out the open ports: #!/usr/bin/perl use strict; use warnings; use HTML::TableExtract; ..."

New page

==Results Scraping==

Here is a simple Perl script (scrape.pl) which dumps out the open ports:

#!/usr/bin/perl
use strict;
use warnings;
use HTML::TableExtract;

my $scan = "";

{
local $/=undef;
open FILE, "ipv6.html" or die "Couldn't open ipv6 file: $!";
$scan = <FILE>;
close FILE;
}
my $te = new HTML::TableExtract();
$te->parse($scan);

# my $pingtable = $te->table(0,0);
my $porttable = $te->table(0,1);

# Currently prints out the OPEN ports
foreach my $row ($porttable->rows)
{
foreach my $ele ( @$row )
{
# Port 7 = RFSD
if (defined $ele)
{
if ($ele =~ m/Port\s+(\d+)\s+=\s+([A-Z]+)/)
{
my $port = $1;
my $state = $2;
if ($state =~ m/OPEN/)
{
printf("PORT %5d was %s\n",$port,$state);
}
}
}
}
}

== Typical Scraped results==

$ ./scrape.pl
PORT 22 was OPEN
PORT 25 was OPEN
PORT 80 was OPEN
PORT 443 was OPEN
$

----

<adsense>1</adsense>

@@ Line 94: / Line 94: @@
+   $
-This website publishes a Privacy Policy. Continued use of this website implies your consent to the use of data outlined in the policy.
+This website publishes a Privacy Policy (link at the bottom of every page). Continued use of this website implies your consent to the use of data outlined in the policy.
 ----

@@ Line 94: / Line 94: @@
+   $
-This website publishes a rivacy Policy. Continued use of this website implies your consent to the use of data outlined in the policy.
+This website publishes a Privacy Policy. Continued use of this website implies your consent to the use of data outlined in the policy.
 ----

← Older revision		Revision as of 15:27, 11 September 2016
Line 1:		Line 1:
−	An example perl script to scrape results from [https://github.com/timsgit/ipscan ~~ipscan~~] web sessions.	+	An example perl script to scrape results from [https://github.com/timsgit/ipscan IPscan] web sessions.

	==Results Scraping==		==Results Scraping==

@@ Line 1: / Line 1: @@
 An example perl script to scrape results from [https://github.com/timsgit/ipscan ipscan] web sessions.
 ==Results Scraping==
@@ Line 96: / Line 94: @@
+   $
-This website publishes a [http://ipv6.chappell-family.com/html/privacy_policy.html Privacy Policy.] Continued use of this website implies your consent to the storage of data outlined in the policy.
+This website publishes a rivacy Policy. Continued use of this website implies your consent to the use of data outlined in the policy.
 ----

← Older revision		Revision as of 07:58, 25 April 2015
Line 99:		Line 99:

	----		----
−
−
−	~~<adsense>1</adsense>~~

← Older revision		Revision as of 16:34, 30 March 2013
Line 1:		Line 1:
−	An ~~introduction~~ to ~~scraping~~ results from ipscan web sessions.	+	An example perl script to scrape results from [https://github.com/timsgit/ipscan ipscan] web sessions.

	<analytics uacct="UA-27654202-1" ></analytics>		<analytics uacct="UA-27654202-1" ></analytics>

← Older revision		Revision as of 13:30, 23 March 2013
Line 16:		Line 16:

	{		{
−	local $/=undef;	+	local $/=undef;
−	open FILE, "ipv6.html" or die "Couldn't open ipv6 file: $!";	+	open FILE, "ipv6.html" or die "Couldn't open ipv6 file: $!";
−	$scan = <FILE>;	+	$scan = <FILE>;
−	close FILE;	+	close FILE;
	}		}
	my $te = new HTML::TableExtract();		my $te = new HTML::TableExtract();
	$te->parse($scan);		$te->parse($scan);

−	# my $pingtable = $te->table(0,0);	+	my $pingtable = $te->table(0,0);
−	my $~~porttable~~ = $te->table(0,1);	+	my $udpporttable = $te->table(0,1);
		+	my $tcpporttable = $te->table(0,2);
		+
		+	# Ping result
		+	foreach my $row ($pingtable->rows)
		+	{
		+	foreach my $ele ( @$row )
		+	{
		+	# <TD TITLE="IPv6 ping">ICMPv6 ECHO REQUEST returned : </TD><TD style="background-color:yellow">ECHO REPLY</TD>
		+	if (defined $ele)
		+	{
		+	if ($ele =~ m/ECHO(.*)REPLY$/)
		+	{
		+	printf("PING response was %sREPLY\n",$1);
		+	}
		+	}
		+	}
		+	}

	# Currently prints out the OPEN ports		# Currently prints out the OPEN ports
−	foreach my $row ($~~porttable~~->rows)	+	foreach my $row ($udpporttable->rows)
	{		{
−	foreach my $ele ( @$row )	+	foreach my $ele ( @$row )
−	{	+	{
−	# Port 7 = RFSD	+	# Port 7 = RFSD
−	if (defined $ele)	+	if (defined $ele)
−	{	+	{
−	if ($ele =~ m/Port\s+(\d+)\s+=\s+([A-Z]+)/)	+	if ($ele =~ m/Port\s+(\d+)\s+=\s+([A-Z]+)/)
−	{	+	{
−	my $port = $1;	+	my $port = $1;
−	my $state = $2;	+	my $state = $2;
−	if ($state =~ m/OPEN/)	+	if ($state =~ m/OPEN/)
−	{	+	{
−	printf("PORT %5d was %s\n",$port,$state);	+	printf("UDP PORT %5d was %s\n",$port,$state);
−	}	+	}
−	}	+	}
−	}	+	}
−	}	+	}
	}		}
		+
		+	foreach my $row ($tcpporttable->rows)
		+	{
		+	foreach my $ele ( @$row )
		+	{
		+	# Port 7 = RFSD
		+	if (defined $ele)
		+	{
		+	if ($ele =~ m/Port\s+(\d+)\s+=\s+([A-Z]+)/)
		+	{
		+	my $port = $1;
		+	my $state = $2;
		+	if ($state =~ m/OPEN/)
		+	{
		+	printf("TCP PORT %5d was %s\n",$port,$state);
		+	}
		+	}
		+	}
		+	}
		+	}
		+


Line 52:		Line 90:

	$ ./scrape.pl		$ ./scrape.pl
−	~~PORT 22~~ was ~~OPEN~~	+	PING response was REPLY
−	PORT 25 was ~~OPEN~~	+	UDP PORT 123 was UDPOPEN
−	PORT 80 was OPEN	+	TCP PORT 80 was OPEN
−	PORT 443 was OPEN	+	TCP PORT 443 was OPEN
−	$	+	$
−		+
	This website publishes a [http://ipv6.chappell-family.com/html/privacy_policy.html Privacy Policy.] Continued use of this website implies your consent to the storage of data outlined in the policy.		This website publishes a [http://ipv6.chappell-family.com/html/privacy_policy.html Privacy Policy.] Continued use of this website implies your consent to the storage of data outlined in the policy.

← Older revision		Revision as of 19:43, 17 February 2013
Line 1:		Line 1:
	An introduction to scraping results from ipscan web sessions.		An introduction to scraping results from ipscan web sessions.
		+
		+	<analytics uacct="UA-27654202-1" ></analytics>

	==Results Scraping==		==Results Scraping==