Perl + Bugzilla in Outreachy Round 13!

I am very proud to be mentoring in the 13th round of Outreachy.

We have a a list of ideas
and a growing list of bugs that would make for good “small contributions”.

I’m open to emails or irc discussions and I’ll try to answer
any and all questions.

email: dylan [at] mozilla [dot] com
or join #bugzilla, I’m in IRC from
13:00 UTC until about 22:00 UTC.

Here’s are project blurb:

Perl is a highly capable, feature-rich programming language with over 28 years of development, making it one of the longest standing FOSS projects. The Perl Foundation is funding a position working on Bugzilla, a widely used, Perl-based issue tracker. In 18 years of development, Bugzilla has grown into a complex application that is used in many different workflows by organizations including Mozilla, the GNOME Project, Red Hat, and Some of this complexity is particularly evident in the search functionality, both in implementation and in user interface. We have several proposals to simplify and improve searching, which will positively impact Bugzilla sites around the world.

Fixed some memory leaks in

So last week I fixed Bug 1282606 which has resulted in a bit of a performance improvement for

Restarts per Hour

is configured to kill processes once they use more than 700M and this happened about every 7 minutes.

About two weeks ago, while working on some performance issues relating to BMO’s new show_bug ui, I discovered that the problem could get worse: running out of memory every 60 seconds. Should everyone switch to the new UI (which is intended to put less load on the server) a lot more load would be on the server. That’s pretty bad, since we want everyone using the new UI as soon as possible. 🙂

This memory leak isn’t new, and I had filed an investigatory bug about it last year. Memory leaks in perl are caused by having cyclic references, and the solution is to not have cycles, use weak references, or to break the cycle when you’re done with whatever data structure it is part of.

I understand the problem, and I know how to fix it… but maybe I don’t know where the problem is?

Thankfully there is a tool for this on CPAN: Devel::MAT.

Using Devel::MAT, it is possible to dump the address space of a perl program and explore it in great detail in a GUI.

I didn’t set out to remove all the memory leaks this time, just the ones that were the biggest or grew the fastest. This meant the TrackingFlags extension Flag objects, the Bug object, and the Comment object.

The changes are on github for the curious,
and the resulting charts below speak for themselves.

Average Request Time

Requests Before Restart

Age of Process Before Restart

Bugzilla on heroku? And hacking on Memcached::libmemcached

I thought I’d take this long weekend to do something fun: get Bugzilla running under Heroku.


  1. fork the perl/psgi buildpack to run bugzilla.
    bugzilla buildpack
  2. swear at Gerv for adding login name support and swear at for spinning in an infinite loop when it can’t prompt for a missing config Bug 1284021.
  3. start writing support for storing the “params” data in the db instead of the filesystem, as the filesystem in heroku is ephermal.
  4. realize that you’ll want to memcache this, so might as well add memcache to heroku
  5. realize MemCachier (one of the herkoku memcached providers) requires username and passwords and that the perl bindings don’t support this

After some research… I realize this feature requires support for the binary protocol and is based on SASL. Fine. I’ll learn XS (perl’s FFI) and contribute code to Memcache::libmemcached. How hard can it be?

It turned out to be not very hard
but it’s a work in progress (and definitely leaks memory right now).

Oh yeah, and bugzilla does run on heroku, but it won’t be useful until the params stuff can be stored in the DB.

Developing Bugzilla with Plackup and cpanminus

(note: these notes will be incorporated into bugzilla documentation after some revisions)
(updated: Fri, May 13 2016 at 10:24 US/Eastern with additions from dkl)

Firstly, we need some system dependencies:

System Dependencies for Debian and Ubuntu users

$ sudo apt-get update
$ sudo apt-get install git perl-modules build-essential cpanminus 
   libssl-dev libexpat1-dev

System Dependencies for CentOS users


Bugzilla Code & CPAN Modules

A checkout of bugzilla is required, and everything we do
will be from that directory.

$ git clone
$ cd bugzilla

Next, we can run with its fancy new –cpanm flag to install all our dependencies into local directory. Note that we do not need to be root for any of these commands, and nothing (from this point on) needs to be installed to the system.

$ perl --cpanm

If all goes well, that should complete with a message saying you need to edit localconfig and re-run checksetup. If you’re fine just using SQLite, you do not need to edit localconfig. Just re-run, as below:

$ perl

The above will prompt you for an email, username, realname and password. After you provide those details, it will configure the database.

Now we can run a development server, with the following perl command below:

$  perl -Ilocal/lib/perl5 local/bin/plackup -R Bugzilla -R -R templates app.psgi

This will start an http on http://localhost:5000.

Navigate to it, and login using the email/password you used above.

Bugzilla will direct you to set the urlbase – which you can set to to the same “http://localhost:5000” above.

Any changes made to files in Bugzilla and templates will trigger a restart. Currently changes to .cgi files might not be picked up, but that should be solved shortly.

Note that these are draft instructions, I appreciate testing and feedback on them. I will fill the todo items in as time allows, and hopefully we get this into the documentation for the Bugzilla 6.0 release.

mom’s birthday

Today would be my mom’s 64th birthday.
I dedicate each Dec 29th to renewing my GPG keys (it is the day my GPG keys would expire if I take no action).

I celebrate mom’s birthday in this way because after one of the (many) times I’ve lost key material (lost in the sense of “the private key is gone” or “I forget the passphrase”) mom offered to keep printouts of said secrets. Unfortunately, mom understood that these things should be well-hidden, and I have yet to figure out the combination of places the various cut-out pieces of paper were hidden… so the key I use now was created after her passing.

Nevertheless, my mom was an amazing lady and I miss her every day.

(Photo: Mom looking contemplative on her 60th birthday)

the following changes have been pushed to

the following changes have been pushed to

  • [1220734] Use of uninitialized value $proxies[0] in string eq at Bugzilla/ line 315.
  • [1219750] Allow Apache2::SizeLimit to be configured via params
  • [1209625] Error when using bzexport: abort: REST error on PUT to The requested method ‘Bug.update’ was not found.“
  • [1177911] Determine and implement better password requirements for BMO
  • [1221851] Parameter description for "password_complexity” is incorrect
  • [1196743] Fix information disclosure vulnerability that allows attacker to obtain victim’s GitHub OAuth return code

discuss these changes on

Reasons why it’s Irish, not Gaelic


  1. Everyone in Ireland calls it Irish. 
  2. Irish is in the Gaelic language family. Calling Irish ‘Gaelic’ would be like calling English ‘West Germanic’.Makes it very difficult to distinguish from other languages of the same family.
  3. Now here’s the kicker. Whether intentional or not, calling Irish ‘Gaelic’ promotes the idea that it is an ancient language and that it isn’t or that it shouldn’t be used today. It suggests that it is the language of the ancient Gaelic people, not of modern Irish people; as if it should be resigned to history. This is not true. Irish is most certainly a modern language. 

Mar sin, stad ar an bpointe boise le bhur dtoil.