Bugzilla Project Updates

As you know, we’re working on two fronts in Bugzilla Development:

A quality-of-life release (5.2) which includes support for utf8mb4,

MySQL 8 compatibility, and compatibility with the latest versions of perl.

A feature-packed release (6.0) which includes the UX/UI from bugzilla.mozilla.org, based on mojolicious.

The 5.2 Update

At the behest of justdave I’ve focused my attention on the quality of life release (5.2). I had desired to be able to release that this week, but this has fallen short because of unforeseen complications.

Here’s a list of things that are ready in the 5.2 branch:

  • MySQL 8 regexp compatibility (it turns out our default emailregexp wasn’t compatible with MySQL 8)
  • SQLite works again (sqlite was broken and is untested in the 5.2 branch)
  • Fixing Safe.pm bug in latest perl (the latest perl has a bug with Safe and Bugzilla must work around it)
  • checksetup.pl completes and large parts of the code work on MySQL 8

The problems remaining are that it is there are many places where snippets of sql are generated and it is non-obvious where and when to quote them. There are cases where an ORDER BYmust be quoted, for instance.

Because the patch set to quote all occurrences of is already quite large, I began exploring a more comprehensive fix on February 3rd. This approach is promising, and may result in a bit of reusable code useful to other Perl applications that have forbidden column or table names that used to work.

At the time of writing, I have written a mysql expression parser that can handle nearly every SQL expression used in Bugzilla. It takes a SQL expression and then quotes all column and table references. While the parser is over 300 lines, it means the overall patch is localized to one new file and a minor change to Bugzilla::DB. I hope to finish the parser this week, and have the patch in review over the weekend (Feb 8/Feb 9).

The 6.0 Update

Since the last meeting, I came up with this (rough) release plan. I’m working on the second version of this, but the gist here is very accurate. We know the “6.0” release will work, because we know bugzilla.mozilla.org works every day. 🙂

  • Delete Mozilla-specific code and branding
    • The Mozilla logo
    • Make it so nobody@mozilla.org is not hard-coded anywhere
    • Remove the Bugzilla::Report::* classes as those are specific reporting features of BMO that
  • Ensure a migration (schema migration) is possible from 5.0 to 6.0.
    • This is mainly a matter of reversing the “multiple aliases” support that was added in 5.0 but is not going to be present in 6.0.
    • There are complications involving the db schema and how email works that are TBD
  • If possible, some dependencies that are difficult to package (or not maintained well) must be dropped.
    • Mostly this is Data::Password::passwdqc.
    • As we will not add new features, this means forgoing password complexity checks which is actually a good thing as passwdqc rejects perfectly fine randomly generated passwords and people hate it.
  • Validate that we can run against PostgreSQL

An apology for impetuous tweeting

I’d like to apologize to this tweet. The events were playing out, and it felt urgent that I act and so I did. I think now I would have said nothing, but still written the letter I ended up writing. That tweet and a few others1 felt impetuous and not how I would like to present myself.

I’m also aware that I hurt at least one close friend, and I am sorry for that. I will try harder to think before I tweet.

  1. There is one where I insulted someone in a mean way that I deleted, but my mistakes remain published.

Open Letter to European Perl Conference

To the organizers of the European Perl Conference in Riga.

I have been involved in the perl community for 22 years. I feel a moral obligation to ensure people that I have introduced to it can continue to feel welcomed and safe. This letter is not meant to shame anyone, but it is public to provide some level of proof that people in the community care about enforcing codes of conduct.

Understanding

There’s actually two issues at hand. The first one is that a person violated the Standards of Conduct (Code of Conduct) at the last Perl Conference in Pittsburgh and is slated to be a keynote speaker for the Perl Conference in Riga.

UPDATE: The keynote speaker issue has been resolved. The rest of this letter remains accurate.

The second issue is that after some prominent people raised concern, there was a perhaps hastily-written blog post that ended with a transphobic joke. The blog post was subsequently edited to remove the joke, along with some tweets relating to the issue.

Adding to the first issue, some may question the acceptability of deadnaming if the victim does not care. It would appear that is still a violation of Standards of Conduct.

Some have already pointed out the bad optics around this, and I’m not here to talk about that. I do care about how this looks to the world at large. This is an open letter precisely because handling this issue in secret would be worse than talking about it.

We’re going to cover Safety and Gender, and at the end some ways which we can resolve this issue. I am not the best person to cover these topics, but I am doing so because that lifts the burden from members of the minority having to explain again and again why something hurt because people want facts.

Safety

People will report Code of Conduct violations. The important question is, how will you handle those incidents and enforce your CoC?

Sage Sharp, 2016-01-25

The underlying issue is one of Safety. Vulnerable individuals attending conference must have faith that the code of conduct will be upheld. No one should be above reproach, and if a person violates the code at an event, it should inform decisions towards that person at other events.

This does not necessarily mean that the person need to be barred from the event — I do not hold an opinion on that matter — but some obviously feel that having the person be a keynote speaker does send a signal that perhaps the conference does not take ensuring the safety of attendees seriously.

You do not have to agree with me on this issue, but you also cannot argue about what trade-offs people make with their personal safety.

For additional information about safety, there is no better source than this collection of FAQs about Codes of Conduct

Gender

Assuming you’re still with me, you agree that people need to feel safe. But you’re struggling because you don’t think the behavior was serious enough.

we’re not talking about serious stuff! He just used the wrong name and pronouns!

some random internet person

This belief is false. Misgendering trans people causes harm.

In the AP News article Misgendering is not a lightweight ‘mistake’ Karolyn Wilson explains that empathy can inform sympathies:

I can’t speak for transgender men and women, but empathy can inform my sympathies: if I feel insulted and demeaned when I am misgendered, how much worse is it for someone who has had to work so much harder than I have to make their outsides match their insides, for someone who is in so much more danger of being discounted as a person or persecuted for who they are?

Karolyn Wilson, Misgendering is not a lightweight ‘mistake’

If you’re reading this and you’re cisgender, presumably you can think back to a time when you were misgendered. If that is not the case, perhaps imagine what it would feel like.

These situations, imagined or otherwise to this tend to have reactions that fall somewhere in a spectrum, but with typical reactions being:

  • You could feel as Karolyn Wilson did — as less of a person, less solid, and less seen
  • Or perhaps you will feel as I did when I first thought about this experience: Indifferent.

If you feel indifferent, and are not yet capable of understanding why gender identity is important you must start believing the lived experience of trans people. You should watch this video by Vi Hart (5 minutes). A quote from this video resonated with me when I first saw it:

My condescending teenager attitude came from a false belief that other people are basically like me.

Vi Hart

I think watching that video and internalizing the line of reasoning Vi Hart uses can help those that are cisgender but not strongly attached to the concepts or expectations of gender.

Now at this point, if you’re still reading I hope you’re with me. Perhaps you’ve read to this point, but you take issue with the joke being characterized as transphobic.

The reason people had a negative reaction to the comment about the perl conference identifying with a different name is because the conference is not a person and drawing this absurd parallel is an attack. This is quite similar to saying ”I identify as an attack helicopter” which is also transphobic and an attack on the concept of gender identity.

There isn’t space to go into this, but this video about gender and this video transphobia are not very long and cover things much better than I can.

I guess I’m done with this topic. As after this some people may label me as a SJW, I’ll pre-emptively tell you to watch The Straight White Man’s Guide to Feminism and Social Justice.

Resolution

Right now people are upset. Several prominent people have called for a boycott of the conference. I’m sure this is not intended. Organizing a conference is very hard, it’s difficult to get the right kinds of help. It’s very stressful and then this happens and people are making demands and I know it must feel like you’re being attacked.

When people say things like “I am appalled by something you have done” it doesn’t mean they are attacking you. It coveys some amount of surprise, and they’re telling you this because they think you are better than the action and can decide to do something different.

Let’s walk it back. There are concrete steps that can be taken to alleviate this problem, and in fact shine.

  1. The SoC violating keynote speaker cannot be a keynote speaker.
  2. An apology for the removed blog post. It was a natural mistake — people will easily forgive such as thing if the apology is genuine.
  3. A commitment to safety must be made. It is not apparent if this conference has a Code of Conduct. The Glasgow conference did, but I don’t see this mentioned initially but I did not look very closely.

happy bmo push day

release tag

the following changes have been pushed to bugzilla.mozilla.org:

  • [1498206] Replace LWP::UserAgent with Mojo::UserAgent in phabbugz extension
  • [1497487] Use HMAC to generate tokens and sensitive graph filenames
  • [1498436] Move site-wide message to global header
  • [1498362] Shutter the “Powered By Mozilla” form
  • [1009716] Add (Cmd|Ctrl)+Enter shortcut for submitting from text areas.
  • [1499262] Bugzilla::DB should gracefully handle disconnection events that happen during transactions
  • [1497077] Convert links to absolute path
  • [1499417] Change BMO docs links from bmo.readthedocs.org to .io
  • [1499269] Refactor common parts of the feed daemon and improve timeout logging
  • [1496004] Improve layout of attachment detail page, hide comment form when custom form is inserted
  • [1495741] Multiple fixes for issues related to memory usage + configuration options for managing memory usage
  • [1482644] Improve “Too many requests” page with an explanation
  • [1499477] Feature request: link from user profile to editusers.

discuss these changes on mozilla.tools.bmo.

happy bmo push day – mojolicious edition

As previously announced at FOSDEM 2018 and then re-announced at MojoConf, bugzilla.mozilla.org is now running on Mojolicious “A next generation web framework for the Perl programming language”

This release incorporates 28 changes and the Mojolicious migration is the least interesting to the end-user, but it is pretty important in terms of being able to deliver rich experiences moving forward.

As an aside, it’s very possible to just download and run bugzilla now,
and the Bugzilla Harmony initiative could use some help cranking out a first release. Interested parties can reach out to me or find us in #bugzilla on irc.mozilla.org

Continue reading “happy bmo push day – mojolicious edition”

happy bmo push day!

release tag

the following changes have been pushed to bugzilla.mozilla.org:

  • [1467297] variable masks earlier declaration in Feed.pm in Phabbugz extension
  • [1467271] When making a revision public, make the revision editable only by the bmo-editbugs-team project (editbugs)
  • [1456877] Add a wrapper around libcmark_gfm to Bugzilla
  • [1468818] Re-introduce is_markdown to the longdescs table (schema-only)
  • [1469689] Remove Bugzilla Helper and custom bug entry form links from Browse page
  • [1419971] Add new Developer Tools and WebExtensions products to easy product selector on Browse and Enter Bug pages
  • [1469827] The etiquette check on “Create new a Bugzilla account” lacks a proper label
  • [1469920] Update schema: add a nickname to profiles table and a fulltext index on the profiles realname field
  • [1469333] Check attachment file size client-side and inform user of too large file before uploading it
  • [1461379] API DB Availability Exceptions on recurring BMO scripts
  • [1393146] Automate blocking IPs that bugzilla flags as exceeding rate limits
  • [1470275] Copy Summary button should give some feedback
  • [1470343] GitHub PR diff is not decoded in UTF-8
  • [1470485] Create new policies using PhabricatorProjectsAllPolicyRule instead of PhabricatorProjectsPolicyRule
  • [1469881] Patches posted by Phabricator to Bugzilla don’t list the patch author
  • [1457900] When restricting a revision to a bugzilla group we should tag the revision with the project
  • [1471044] Allow some model classes to have dynamic column names with class method DYNAMIC_COLUMNS
  • [1470966] “Status” column in Phabricator dashboard isn’t very useful
  • [1452096] Some custom dropdown UI widgets stay fixed and don’t move with scroll
  • [1471304] Block sending mail to hosts that end with .tld or .bugs
  • [1457550] Update scripts/remove-non-public-data.pl suitability for current BMO infrastructure.
  • [1469023] Show “new changes since (datetime)” indicator that links to unread changes/comments

discuss these changes on mozilla.tools.bmo.

Changes to Secure Bugmail on bugzilla.mozilla.org

There’s a big change coming on May 16th, 2018:

We’ve replaced the encryption code for secure bugmail.
All OpenPGP-formatted bugmail will be encrypted using the openpgp.js library.
There are no changes to the S/MIME encryption, and if you’re already using S/MIME my recommendation is to continue using it.

There are upsides (such as a new feature and several bugfixes) and only a single downside to this change.

Upsides

  • Feature: Messages may be encrypted using the Elliptic curves P-256, P-384, P-521, SECP-256k1, Curve25519, and Ed25519 (You can generate ECC keys with the command gpg --expert --full-gen-key and choosing option 9)
  • Fix Bug 790487: Messages will be encrypted to subkeys when possible.
  • Fix Bug 1190749: Messages will be encrypted using AES256, instead of CAST 5
  • Fix Bug 1256321: Messages will not be encrypted with expired keys.

Future Upsides

In addition to these changes, future work may allow:

  • Generating and receiving wild card key IDs in public-key encrypted session key packets.
  • Experimental opt-in authenticated encryption (AES-EAX, OCB, or GCM) based on the IETF proposal

The Downside

There are a few users whose keys will not work for various reasons.
If you’re one of those users, you can expect an email today (May 11th) explaining your options.

Questions

In anticipation of questions that may be asked, here are some answers.

Why OpenPGP.js?

  • Using gpg is difficult because its API is based on executing processes and communicating over 4 (or perhaps more) file descriptors.
  • Using gpg is also stateful because it must maintain its own key database.
  • Our existing OpenPGP library, while considered the second-most-complete OpenPGP implementation, hasn’t been actively maintained in a while. ProtonMail is actively maintaining OpenPGP.js since 2016, and this inspires confidence.
  • OpenPGP.js has undergone two complete security audits from Cure53. The first audit is available for review.

Will my GPG key work?

Probably. If not, you’ll be hearing from me.

If you would like to check your own key, you can use this baroque single page app to see what keys OpenPGP.js supports.

happy bmo push day!

release tag

the following changes have been pushed to bugzilla.mozilla.org:

  • [1450325] Update email templates with instructions for unsubscribing from all emails
  • [1451599] Checkbox for agreement terms at create account page should be on the left side
  • [1438205] Preserve comments in progress across page reloads
  • [1452531] PhabBugz code should add allow visibility to reviewers when creating custom policies
  • [1440828] Phabricator review requests should show up on the BMO dashboard
  • [1452241] Improve feed error handling and logging
  • [1453124] extensions/PhabBugz/bin/update_project_members.pl should be combined with the normal feed daemon
  • [1455493] cleanup push connector logging
  • [1427395] Allow request_cache to be constant-folded in Bugzilla.pm
  • [1455772] Label bug bounty form credit fields
  • [1373280] Highlight private comments in new bug modal UI
  • [1430367] Ssl preconnect google analytics
  • [1456529] Support SameSite attribute on session cookies
  • [1441732] Improve missing module error in Bugzilla::Extensions and catch more compile errors in tests
  • [1457031] When a revision does not have an bug id, the bug is made public but we also need to remove secure-revision tag

discuss these changes on mozilla.tools.bmo.