Bigger and better search box on BMO

There is a lot of power hidden in quick search and my data suggests it is under-utilized.

For instance, searching for all review flags is the literal search tag review?
Similarly you can do needinfo?dylan@mozilla.com to find all bugs with needinfos directed at me (actually, this query performs a slightly broader search but I have code to fix that).

There are dozens more examples. The quick search help is a long read, and most people won’t bother.

A long time ago glob suggested stealing the UI from DXR, where you get a little quick-help on the operator
syntax for DXR searches. It’s a pretty simple change, right?

Well, our search box is small. So the first thing it needs is to be bigger.

bigger quicksearch box

More room to work with. This required replacing the table-based layout with some flex boxes. The top-line is nearly pixel perfect
to its previous table-based implementation.

We can also hide some things and begin making the UI responsive

portrait view of bmo quicksearch.

I hope to post a followup showing the quicksearch syntax helper,
but this is at the moment just a side task.

(Although it ties well into the goal of implementing elasticsearch on BMO).

Sorry, I meant I changed it from 226s to 184ms

About twenty days ago it came to my attention via my colleague Ed Morley that BMO’s bzapi was very slow.
It turns out he had reported the same issue the prior year as well!

Performance problems are very enjoyable to work on, I find. Especially when they are reproducible.

I lost most of the day on this, but in the end I was able to take the slowest function from executing at a leisurely 226 seconds to a very fast 184 milliseconds

Perl + Bugzilla in Outreachy Round 13!

I am very proud to be mentoring in the 13th round of Outreachy.

We have a a list of ideas
and a growing list of bugs that would make for good “small contributions”.

I’m open to emails or irc discussions and I’ll try to answer
any and all questions.

email: dylan [at] mozilla [dot] com
or join irc.mozilla.org #bugzilla, I’m in IRC from
13:00 UTC until about 22:00 UTC.

Here’s are project blurb:

Perl is a highly capable, feature-rich programming language with over 28 years of development, making it one of the longest standing FOSS projects. The Perl Foundation is funding a position working on Bugzilla, a widely used, Perl-based issue tracker. In 18 years of development, Bugzilla has grown into a complex application that is used in many different workflows by organizations including Mozilla, the GNOME Project, Red Hat, and freedesktop.org. Some of this complexity is particularly evident in the search functionality, both in implementation and in user interface. We have several proposals to simplify and improve searching, which will positively impact Bugzilla sites around the world.

Fixed some memory leaks in bugzilla.mozilla.org

So last week I fixed Bug 1282606 which has resulted in a bit of a performance improvement for bugzilla.mozilla.org:

Restarts per Hour

Apache2::SizeLimit
is configured to kill processes once they use more than 700M and this happened about every 7 minutes.

About two weeks ago, while working on some performance issues relating to BMO’s new show_bug ui, I discovered that the problem could get worse: running out of memory every 60 seconds. Should everyone switch to the new UI (which is intended to put less load on the server) a lot more load would be on the server. That’s pretty bad, since we want everyone using the new UI as soon as possible. 🙂

This memory leak isn’t new, and I had filed an investigatory bug about it last year. Memory leaks in perl are caused by having cyclic references, and the solution is to not have cycles, use weak references, or to break the cycle when you’re done with whatever data structure it is part of.

I understand the problem, and I know how to fix it… but maybe I don’t know where the problem is?

Thankfully there is a tool for this on CPAN: Devel::MAT.

Using Devel::MAT, it is possible to dump the address space of a perl program and explore it in great detail in a GUI.

I didn’t set out to remove all the memory leaks this time, just the ones that were the biggest or grew the fastest. This meant the TrackingFlags extension Flag objects, the Bug object, and the Comment object.

The changes are on github for the curious,
and the resulting charts below speak for themselves.

Average Request Time

Requests Before Restart

Age of Process Before Restart

Bugzilla on heroku? And hacking on Memcached::libmemcached

I thought I’d take this long weekend to do something fun: get Bugzilla running under Heroku.

Steps:

  1. fork the perl/psgi buildpack to run bugzilla.
    bugzilla buildpack
  2. swear at Gerv for adding login name support and swear at
    checksetup.pl for spinning in an infinite loop when it can’t prompt for a missing config Bug 1284021.
  3. start writing support for storing the “params” data in the db instead of the filesystem, as the filesystem in heroku is ephermal.
  4. realize that you’ll want to memcache this, so might as well add memcache to heroku
  5. realize MemCachier (one of the herkoku memcached providers) requires username and passwords and that the perl bindings don’t support this

After some research… I realize this feature requires support for the binary protocol and is based on SASL. Fine. I’ll learn XS (perl’s FFI) and contribute code to Memcache::libmemcached. How hard can it be?

It turned out to be not very hard
but it’s a work in progress (and definitely leaks memory right now).

Oh yeah, and bugzilla does run on heroku, but it won’t be useful until the params stuff can be stored in the DB.