Nearly a month ago, I got back from this year’s QA hackathon in Birmingham, UK. This is a little late for my summary, so I’ll also write a bit about progress since the hackathon. This year my work focused on CPAN Testers, the related Metabase framework, and MYMETA.yml support for the toolchain.
First, I’d like to thank Birmingham.pm for sponsoring my trip. The Birmingham.pm team organized a great event. Perhaps in a future post, I’ll write more about some of the things that make for a good hackathon. But I think that Oslo.pm and Birmingham.pm have set a high bar for future organizers.
CPAN Testers & Metabase 🔗︎
I spent most of my time at the hackathon working on “CPAN Testers 2.0” (CT 2.0), picking up from where Ricardo and I left off from last year’s hackathon.
Currently, CPAN Testers uses email to collect reports and makes them available via NNTP. With over 3.5 million reports sent to date and monthly volumes now exceeding 200,000 reports, this puts a strain on the perl.org servers. Moreover, many would-be testers aren’t able to send reports due to network restrictions on outbound email.
To overcome these and other flaws, CT 2.0 is intended to have:
- A web service to submit, search and retrieve reports
- Reports as structured data, not just text
- A single client with unified “grading” logic for CPAN.pm, CPANPLUS or other tools
At the hackathon: A team effort 🔗︎
Last year, Ricardo and I sketched out a preliminary design for “Metabase”, a web-service/database framework that will be the new back-end for CT 2.0. This year, we enlisted several others to work on components of the system or provide constructive criticism on the design. New contributors included Barbie, Leon Brocard, Rich Dawe, Tux, Andreas Koenig, Chris Williams, and Jos Boumans. ((If I’ve forgotten anyone or missed any contributions below, I apologize in advance. ))
On the client side, I focused most on defining the structured format for CPAN Testers reports in terms of Metabase::Fact classes and worked with Ricardo to hammer out a model for managing user identities. I also worked with Jos and Chris to sketch out the API for the future, common CPAN Testers client and the interface to CPANPLUS.
Meanwhile, Andreas and Tux created Config::Perl::V to extract configuration information for reports. Rich Dawe wrote a spike implementation of Test::Reporter::Transport::Metabase, which will be used to send legacy reports to a Metabase backend. Incredibly, Rich was able to demonstrate it working with dummy data despite having the metabase client, the data model and the web service API all in flux throughout the hackathon.
On the server side, Ricardo iterated the guts of the Metabase a few times as our team raised and then resolved various design flaws. Leon implemented new Metabase::Archive and Metabase::Index classes based on SQLite and Solr, respectively. Barbie defined the search semantics the Metabase would need to drive CPAN Testers analytics and notification services and more generally played design coach to ensure consistency with CT 1.0.
After the hackathon: It works! 🔗︎
One thing that bothered me about the hackathon was that we never actually got to the point of submitting a “real” CPAN Testers report into a metabase. We were very, very close, but just ran out of time. Getting it done required a bit of work to adapt Rich’s Transport class to the final form of the Metabase and CPAN Testers libraries and adding support for user identities.
With that done, I’ve been able to take saved Test::Reporter reports from CPAN::Reporter and inject them into a locally-running Metabase. It works! This means that we’re not far (in lines-of-code at least) from being able to have today’s testers submit legacy style reports into a real Metabase somewhere.
The next thing on my agenda is working on the common client for CPAN.pm and CPANPLUS. Then I plan to work on importing the 3.5 million legacy reports into a Metabase and see how it scales. Meanwhile, I’ll be pushing others ((You know who you are!)) to get a robust Metabase server running on cpantesters.org.
If you are interested in participating, the CT 2.0 and Metabase modules aren’t yet released, but are available on github.com:
- Metabase
- Metabase::Fact
- Metabase::Client::Simple
- Metabase::Web
- CPAN::Testers::Report
- Test::Reporter::Transport::Metabase
Would-be developers should also join the cpan-testers-discuss mailing list ((Email cpan-testers-discuss-subscribe@perl.org)) and the metabase mailing list. We’re also usually found on IRC on #toolchain at irc.perl.org.
On a somewhat related topic, I also released Test::Reporter 1.53_03 with fixes for SMTP line length problems that were corrupting reports for some testers. It also adds some hooks that will be helpful to Test::Reporter::Transport::Metabase.
MYMETA.yml 🔗︎
In a few spare moments. I also added support for MYMETA.yml ((Previously called METALOCAL.yml)) to a branch of the Module::Build repository. MYMETA.yml was proposed at last year’s hackathon as a standard method for Perl toolchain modules to communicate post-configuration dependencies (i.e. after Makefile.PL or Build.PL have run) using the standard META.yml format that many toolchain modules already support.
As Perl 5.10.1 is (hopefully) right around the corner, MYMETA.yml won’t be added to Module::Build until after 5.10.1 is released. In the meantime, I merged support for MYMETA.yml into the CPAN.pm master development branch, so CPAN.pm will be ready for MYMETA.yml when build tools start generating them. The “plan” (to the extent there is one), is to get MYMETA.yml support into CPAN.pm and CPANPLUS first, then update the various build tools to generate it.