Migrate all issues from Google Code to github [rt.cpan.org #104912]

[oalders]

Migrated from rt.cpan.org#104912 (status was 'open')

Requestors:

• [email protected]
• [email protected]

From [email protected] on 2015-06-03 17:05:30:

Google Code is shutting down: http://google-opensource.blogspot.ca/2015/03/farewell-to-google-code.html

This is where the "official" bug tracker for this distribution is located. We need to migrate all these tickets.

Since there is a programmatic way of migrating issues from Google Code to github, and this distribution is already managed by the libwww-perl group in github, I propose that the tickets be migrated to there rather than to RT.

I am seeing now about getting the issue queue opened up at https://github.com/libwww-perl/WWW-Mechanize.

From [email protected] on 2015-06-09 11:05:13:

s/list/link/   in the title of this report.

From [email protected] on 2015-06-09 16:45:33:

On 2015-06-09 04:05:13, [email protected] wrote:
> s/list/link/   in the title of this report.


This is a duplicate of #104912.

From [email protected] on 2015-07-20 19:19:34:

On 2015-06-03 10:05:30, ETHER wrote:
> Google Code is shutting down: http://google-opensource.blogspot.ca/2015/03/farewell-to-google-code.html
> 
> This is where the "official" bug tracker for this distribution is
> located. We need to migrate all these tickets.
> 
> Since there is a programmatic way of migrating issues from Google Code
> to github, and this distribution is already managed by the libwww-perl
> group in github, I propose that the tickets be migrated to there
> rather than to RT.
> 
> I am seeing now about getting the issue queue opened up at
> https://github.com/libwww-perl/WWW-Mechanize.


I'm raising this issue to critical status now as August is the deadline for making any changes to the Google Code repository and issue queue. WE NEED TO GET THIS CONTENT MIGRATED ASAP.

Note that only the original maintainers apparently have access now. I tried to get a Pull Request Challenge participant to look at the migration last month, but access is locked down so she was not able to (nor do I have access).

From [email protected] on 2015-07-20 19:59:50:

Is there something you need from me?

From [email protected] on 2015-07-20 20:57:14:

On 2015-07-20 12:59:50, PETDANCE wrote:
> Is there something you need from me?

Someone with access to the Google Code repository (i.e. not me) needs to do the migration. See the first message in this ticket.

From [email protected] on 2015-07-20 21:06:36:

I don't think the Google Code repo even exists any more.  I'm not seeing it.

From [email protected] on 2015-07-20 21:53:27:

On 2015-07-20 14:06:36, PETDANCE wrote:
> I don't think the Google Code repo even exists any more.  I'm not seeing it.

That's really unfortunate. I remember it existing last year, with quite a number of tickets in the issue queue...

From [email protected] on 2015-07-21 14:33:13:

The Wayback Machine has a snapshot as of December 2014.  The vast majority of the tickets, from what I see from skimming through them, are just cut & pastes from RT anyway, back to 2007.

I suggest that a mass copy of tickets over to GitHub would be anti-helpful, anyway.  This seems like it would be a good time to weed out ancient tickets that are never going to get addressed.

I would be glad to work with someone to get the tickets triaged and moved over to GitHub.

[petdance]

https://web.archive.org/web/20150227111857/http://code.google.com/p/www-mechanize/issues/list

[spazm]

Looking into this.

• scrape and de-wayback-machinize the issues archived by the WaybackMachine at web.archive.org
• parse and extract the individual issues
• import into github issues. Q: How to tag them?

I've scraped and de-wayback-machined the issues from the 2015 wayback link (using WWW::Mechanize and WWW::WebArchive::WaybackMachine to clean the html).

I'll get the individual issues parsed and extracted soon.

How would you like the tickets? Just bulk import them with appropriate labels? Or I could provide a spreadsheet for triaging?

Snapshot of my code in progress.

#!/usr/bin/env perl

use strict;
use warnings;

use WWW::WebArchive::WaybackMachine;

my $url = "http://code.google.com/p/www-mechanize/issues/list";

# Download specific older wayback urls, since the most
# current versions 301 redirect to an empty page.

my @wayback_urls = (
    "https://web.archive.org/web/20150227111857/$url",
    "https://web.archive.org/web/20130510011416/http://code.google.com/p/www-mechanize/issues/list?num=100&start=100"
);

my $wayback = WWW::WebArchive::WaybackMachine->new(
    url     => $url,
    verbose => 1,
);

# make our own Mech, to use with WWW::WA::WM methods
my $ua = WWW::Mechanize->new();
$ua->agent_alias('Windows IE 6');
$ua->stack_depth(1);

my $file   = $url;
$file      =~ s,https?://,,;
my $dir    = './';
my $domain = 'code.google.com';

for my $wayback_url (@wayback_urls) {
    $ua->get($wayback_url);
    my @links = $ua->find_all_links(
        tag        => "a",
        text_regex => qr/^\d+$/,
        url_regex  => qr/issues.detail/,
    );

    foreach my $link (@links) {
        printf ("text:%s url:%s\n", $link->text, $link->url);
        $wayback->mirror($ua, $link->url, $link->text, $dir, $domain);
    }
}

[spazm]

Nearly ready to load the issues. I've skipped thinking about tagging or assigning the issues, as I don't have the privs necessary in the API.

[spazm]

Finished migrating issues from google code, into new issues #116 through #225.

I've created a repo with the scripts and parsed data here: https://github.com/spazm/google_code_to_github_issues

#CPAN-PRC #TeamZiprecruiter.

Please close this ticket. Thanks.

[oalders]

@spazm thanks so much. This is great to have!