Bugzilla Project Updates

As you know, we’re working on two fronts in Bugzilla Development:

A quality-of-life release (5.2) which includes support for utf8mb4,

MySQL 8 compatibility, and compatibility with the latest versions of perl.

A feature-packed release (6.0) which includes the UX/UI from bugzilla.mozilla.org, based on mojolicious.

The 5.2 Update

At the behest of justdave I’ve focused my attention on the quality of life release (5.2). I had desired to be able to release that this week, but this has fallen short because of unforeseen complications.

Here’s a list of things that are ready in the 5.2 branch:

  • MySQL 8 regexp compatibility (it turns out our default emailregexp wasn’t compatible with MySQL 8)
  • SQLite works again (sqlite was broken and is untested in the 5.2 branch)
  • Fixing Safe.pm bug in latest perl (the latest perl has a bug with Safe and Bugzilla must work around it)
  • checksetup.pl completes and large parts of the code work on MySQL 8

The problems remaining are that it is there are many places where snippets of sql are generated and it is non-obvious where and when to quote them. There are cases where an ORDER BYmust be quoted, for instance.

Because the patch set to quote all occurrences of is already quite large, I began exploring a more comprehensive fix on February 3rd. This approach is promising, and may result in a bit of reusable code useful to other Perl applications that have forbidden column or table names that used to work.

At the time of writing, I have written a mysql expression parser that can handle nearly every SQL expression used in Bugzilla. It takes a SQL expression and then quotes all column and table references. While the parser is over 300 lines, it means the overall patch is localized to one new file and a minor change to Bugzilla::DB. I hope to finish the parser this week, and have the patch in review over the weekend (Feb 8/Feb 9).

The 6.0 Update

Since the last meeting, I came up with this (rough) release plan. I’m working on the second version of this, but the gist here is very accurate. We know the “6.0” release will work, because we know bugzilla.mozilla.org works every day. 🙂

  • Delete Mozilla-specific code and branding
    • The Mozilla logo
    • Make it so nobody@mozilla.org is not hard-coded anywhere
    • Remove the Bugzilla::Report::* classes as those are specific reporting features of BMO that
  • Ensure a migration (schema migration) is possible from 5.0 to 6.0.
    • This is mainly a matter of reversing the “multiple aliases” support that was added in 5.0 but is not going to be present in 6.0.
    • There are complications involving the db schema and how email works that are TBD
  • If possible, some dependencies that are difficult to package (or not maintained well) must be dropped.
    • Mostly this is Data::Password::passwdqc.
    • As we will not add new features, this means forgoing password complexity checks which is actually a good thing as passwdqc rejects perfectly fine randomly generated passwords and people hate it.
  • Validate that we can run against PostgreSQL

Bugzilla Harmony Beta

In addition to the Mojolicious ❤️, we’re also focused on more near-term gains.
Specifically getting the Bugzilla/Harmony branch running under PSGI, and being a thing you can download and use.

I am happy to announce today that the master branch is in a beta-quality state as of today,

At the moment, the following installation scenarios have been tested:

  • checkout the code
  • run cpanm --installdeps --notest --with-feature=bmo -l local .
  • run checksetup.pl and edit localconfig to point your database (only mysql is currently working)
  • run app.psgi, optionally with starman

If you would like to help the project, a good place to start would be testing this on your systems and reporting back findings.

The next milestone will be integration of the Mojolicious work currently going on in PR #517 in the bmo repo

bugzilla harmony news

Thanks to the near-heroic efforts of CyberShadow the unstable harmony branch is getting quite close to being able to run as an independent Bugzilla install.

For a number of reasons, bmo has a separate repo for managing its dependencies. Mostly this is about efficiency — dependencies change less often then the code, and we can have much faster builds and CI.

In a step towards independence I have forked the “bmo-systems” repo as harmony-systems, and with any luck the “unstable” branch of the harmony repo will soon be passing its own tests.

Note that the eventual goal is for the bmo and harmony repositories to be “nearly identical”, differing only in disabled extensions. That said, we need flexibility at times to experiment.

bugzilla/harmony unstable is kinda sorta working

So after staring at a wall for a while, I realized it was lingering code in the ‘BMO’ extension causing my non-working-ness from Saturday.

So current status is that a checkout of bugzilla/harmony‘s unstable branch should be runnable now.

cpanm --with-feature=bmo --installdeps .
perl checksetup.pl

then edit localconfig to point db_host to a mysql db, set urlbase to http://localhost:5000/, and then

perl checksetup.pl
perl app.psgi

Of course, running checksetup a second third is currently broken, but I suspect another round of poking at things and that will work.

TODO

  • re-enable SecureMail (and merge its schema into core
  • disable BMO extension in harmony (and test stuff)
  • make tracking flags optional (maybe; maybe we want it in the core)
  • split out schema migration from checksetup
  • ???

bugzilla harmony saturday update

PSGI seems pretty stable, so now to make the rest actually work!

First I’m moving schema changes out of extensions. This doesn’t pass tests yet — and it is not complete. I’ll have to update templates as well.

Next, a lot of our UX code assumes a workflow that is different than the default values. Ideally we’d support changing the workflow, but for the moment
the default workflow provided should be the one the bmo one.

Finally, we’re still getting an error because the product security group doesn’t yet have a good default. My early attempt this is not entirely working…

From now, whatever the last thing I’ve been working on will be in this unstable branch, because in a week or two I hope to have master in a consistent, release-ready state.

I’ll spend a bit of time Sunday as well, and hope to have something runnable without having to run the scripts/generate_bmo_data.pl hack.