fidzu : firefox

Extern types have been blocked for an unreasonably long time on a fairly narrow, specialized question: Rust today divides all types into two categories - sized, whose size can be statically computed, and unsized, whose size can only be computed at runtime. But for external types what we really want is a third category, types whose size can never be known, even at runtime (in C, you can model this by defining structs with an unknown set of fields). The problem is that Rust's ?Sized notation does not naturally scale to this third case. I think it's time we fixed this. At some point I read a proposal - I no longer remember where - that seems like the obvious way forward and which I think is a win on several levels. So I thought I would take a bit of time to float the idea again, explain the tradeoffs I see with it, and explain why I think the idea is a good change.

TL;DR: write T: Unsized in place of T: ?Sized (and sometimes T: DynSized)

The basic idea is to deprecate the ?Sized notation and instead have a family of Sized supertraits. As today, the default is that every type parameter T gets a T: Sized bound unless the user explicitly chooses one of the other supertraits:

/// Types whose size is known at compilation time (statically).
/// Implemented by (e.g.) `u32`. References to `Sized` types
/// are "thin pointers" -- just a pointer.
trait Sized: DynSized { }

/// Types whose size can be computed at runtime (dynamically).
/// Implemented by (e.g.) `[u32]` or `dyn Trait`.
/// References to these types are "wide pointers",
/// with the extra metadata making it possible to compute the size
/// at runtime.
trait DynSized: Unsized { }

/// Types that may not have a knowable size at all (either statically or dynamically).
/// All types implement this, but extern types **only** implement this.
trait Unsized { }

Under this proposal, T: ?Sized notation could be converted to T: DynSized or T: Unsized. T: DynSized matches the current semantics precisely, but T: Unsized is probably what most uses actually want. This is because most users of T: ?Sized never compute the size of T but rather just refer to existing values of T by pointer.

Credit where credit is due?

For the record, this design is not my idea, but I'm not sure where I saw it. I would appreciate a link so I can properly give credit.

Why do we have a default T: Sized bound in the first place?

It's natural to wonder why we have this T: Sized default in the first place. The short version is that Rust would be very annoying to use without it. If the compiler doesn't know the size of a value at compilation time, it cannot (at least, cannot easily) generate code to do a number of common things, such as store a value of type T on the stack or have structs with fields of type T. This means that a very large fraction of generic type parameters would wind up with T: Sized.

So why the ?Sized notation?

The ?Sized notation was the result of a lot of discussion. It satisfied a number of criteria.

? signals that the bound operates in reverse

The ? is meant to signal that a bound like ?Sized actually works in reverse from a normal bound. When you have T: Clone, you are saying "type T must implement Clone". So you are narrowing the set of types that T could be: before, it could have been both types that implement Clone and those that do not. After, it can only be types that implement Clone. T: ?Sized does the reverse: before, it can only be types that implement Sized (like u32), but after, it can also be types that do not (like [u32] or dyn Debug). Hence the ?, which can be read as "maybe" - i.e., T is "maybe" Sized.

? can be extended to other default bounds

The ? notation also scales to other default traits. Although we've been reluctant to exercise this ability, we wanted to leave room to add a new default bound. This power will be needed if we ever adopt "must move" types¹ or add a bound like ?Leak to signal a value that cannot be leaked.

But ? doesn't scale well to "differences in degree"

When we debated the ? notation, we thought a lot about extensibility to other orthogonal defaults (like ?Leak), but we didn't consider extending a single dimension (like Sized) to multiple levels. There is no theoretical challenge. In principle we could say…

• T means T: Sized + DynSized
• T: ?Sized drops the Sized default, leaving T: DynSized
• T: ?DynSized drops both, leaving any type T

…but I personally find that very confusing. To me, saying something "might be statically sized" does not signify that it is dynamically sized.

And ? looks "more magical" than it needs to

Despite knowing that T: ?Sized operates in reverse, I find that in practice it still feels very much like other bounds. Just like T: Debug gives the function the extra capability of generating debug info, T: ?Sized feels to me like it gives the function an extra capability: the ability to be used on unsized types. This logic is specious, these are different kinds of capabilities, but, as I said, it's how I find myself thinking about it.

Moreover, even though I know that T: ?Sized "most properly" means "a type that may or may not be Sized", I find it wind up thinking about it as "a type that is unsized", just as I think about T: Debug as a "type that is Debug". Why is that? Well, beacuse ?Sized types may be unsized, I have to treat them as if they are unsized - i.e., refer to them only by pointer. So the fact that they might also be sized isn't very relevant.

How would we use these new traits?

So if we adopted the "family of sized traits" proposal, how would we use it? Well, for starters, the size_of methods would no longer be defined as T and T: ?Sized…

fn size_of<T>() -> usize {}
fn size_of_val<T: ?Sized>(t: &T) -> usize {}

… but instead as T and T: DynSized …

fn size_of<T>() -> usize {}
fn size_of_val<T: DynSized>(t: &T) -> usize {}

That said, most uses of ?Sized today do not need to compute the size of the value, and would be better translated to Unsized…

impl<T: Unsized> Debug for &T {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) { .. }
}

Option: Defaults could also be disabled by supertraits?

As an interesting extension to today's system, we could say that every type parameter T gets an implicit Sized bound unless either…

1. There is an explicit weaker alternative(like T: DynSized or T: Unsized);
2. Or some other bound T: Trait has an explicit supertrait DynSized or Unsized.

This would clarify that trait aliases can be used to disable the Sized default. For example, today, one might create a Value trait is equivalent to Debug + Hash + Org, roughly like this:

trait Value: Debug + Hash + Ord {
    // Note that `Self` is the *only* type parameter that does NOT get `Sized` by default
}

impl<T: ?Sized + Debug + Hash + Ord> Value for T {}

But what if, in your particular data structure, all values are boxed and hence can be unsized. Today, you have to repeat ?Sized everywhere:

struct Tree<V: ?Sized + Value> {
    value: Box<V>,
    children: Vec<Tree<V>>,
}

impl<V: ?Sized + Value> Tree<V> { … }

With this proposal, the explicit Unsized bound could be signaled on the trait:

trait Value: Debug + Hash + Ord + Unsized {
    // Note that `Self` is the *only* type parameter that does NOT get `Sized` by default
}

impl<T: Unsized + Debug + Hash + Ord> Value for T {}

which would mean that

struct Tree<V: Value> { … }

would imply V: Unsized.

Alternatives

Different names

The name of the Unsized trait in particular is a bit odd. It means "you can treat this type as unsized", which is true of all types, but it sounds like the type is definitely unsized. I'm open to alternative names, but I haven't come up with one I like yet. Here are some alternatives and the problems with them I see:

• Unsizeable - doesn't meet our typical name conventions, has overlap with the Unsize trait
• NoSize, UnknownSize - same general problem as Unsize
• ByPointer - in some ways, I kind of like this, because it says "you can work with this type by pointer", which is clearly true of all types. But it doesn't align well with the existing Sized trait - what would we call that, ByValue? And it seems too tied to today's limitations: there are, after all, ways that we can make DynSized types work by value, at least in some places.
• MaybeSized - just seems awkward, and should it be MaybeDynSized?

All told, I think Unsized is the best name. It's a bit wrong, but I think you can understand it, and to me it fits the intuition I have, which is that I mark type parameters as Unsized and then I tend to just think of them as being unsized (since I have to).

Some sigil

Under this proposal, the DynSized and Unsized traits are "magic" in that explicitly declaring them as a bound has the impact of disabling a default T: Sized bound. We could signify that in their names by having their name be prefixed with some sort of sigil. I'm not really sure what that sigil would be - T: %Unsized? T: ?Unsized? It all seems unnecessary.

Drop the implicit bound altogether

The purist in me is tempted to question whether we need the default bound. Maybe in Rust 2027 we should try to drop it altogether. Then people could write

fn size_of<T: Sized>() -> usize {}
fn size_of_val<T: DynSized>(t: &T) -> usize {}

and

impl<T> Debug for &T {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) { .. }
}

Of course, it would also mean a lot of Sized bounds cropping up in surprising places. Beyond random functions, consider that every associated type today has a default Sized bound, so you would need

trait Iterator {
    type Item: Sized;
}

Overall, I doubt this idea is worth it. Not surprising: it was deemed too annoying before, and now it has the added problem of being hugely disruptive.

Conclusion

I've covered a design to move away from ?Sized bounds and towards specialized traits. There are avrious "pros and cons" to this proposal but one aspect in particular feels common to this question and many others: when do you make two "similar but different" concepts feel very different - e.g., via special syntax like T: ?Sized - and when do you make them feel very similar - e.g., via the idea of "special traits" where a bound like T: Unsized has extra meaning (disabling defaults).

There is a definite trade-off here. Distinct syntax help avoid potential confusion, but it forces people to recognize that something special is going on even when that may not be relevant or important to them. This can deter folks early on, when they are most "deter-able". I think it can also contribute to a general sense of "big-ness" that makes it feel like understanding the entire language is harder.

Over time, I've started to believe that it's generally better to make things feel similar, letting people push off the time at which they have to learn a new concept. In this case, this lessens my fears around the idea that Unsized and DynSized traits would be confusing because they behave differently than other traits. In this particular case, I also feel that ?Sized doesn't "scale well" to default bounds where you want to pick from one of many options, so it's kind of the worst of both worlds - distinct syntax that shouts at you but which also fails to add clarity.

Ultimately, though, I'm not wedded to this idea, but I am interested in kicking off a discussion of how we can unblock extern types. I think by now we've no doubt covered the space pretty well and we should pick a direction and go for it (or else just give up on extern types).

23 Apr 2024 8:51pm GMT

Firefox's crash reporter is hopefully not something that most users experience often. However, it is still a very important component of Firefox, as it is integral in providing insight into the most visible bugs: those which crash the main process. These bugs offer the worst user experience (since the entire application must close), so fixing them is a very high priority. Other types of crashes, such as content (tab) crashes, can be handled by the browser and reported gracefully, sometimes without the user being aware that an issue occurred at all. But when the main browser process comes to a halt, we need another separate application to gather information about the crash and interact with the user.

This post details the approach we have taken to rewrite the crash reporter in Rust. We discuss the reasoning behind this rewrite, what makes the crash reporter a unique application, the architecture we used, and some details of the implementation.

Why Rewrite?

Even though it is important to properly handle main process crashes, the crash reporter hasn't received significant development in a while (aside from development to ensure that crash reports and telemetry continue to reliably be delivered)! It has long been stuck in a local maximum of "good enough" and "scary to maintain": it features 3 individual GUI implementations (for Windows, GTK+ for Linux, and macOS), glue code abstracting a few things (mostly in C++, and Objective-C for macOS), a binary blob produced by obsoleted Apple development tools, and no test suite. Because of this, there is a backlog of features and improvements which haven't been acted on.

We've recently had a number of successful pushes to decrease crash rates (including both big leaps and many small bug fixes), and the crash reporter has functioned well enough for our needs during this time. However, we've reached an inflection point where improving the crash reporter would provide valuable insight to enable us to decrease the crash rate even further. For the reasons previously mentioned, improving the current codebase is difficult and error-prone, so we deemed it appropriate to rewrite the application so we can more easily act on the feature backlog and improve crash reports.

Like many components of Firefox, we decided to use Rust for this rewrite to produce a more reliable and maintainable program. Besides the often-touted memory safety built into Rust, its type system and standard library make reasoning about code, handling errors, and developing cross-platform applications far more robust and comprehensive.

Crash Reporting is an Edge Case

There are a number of features of the crash reporter which make it quite unique, especially compared to other components which have been ported to Rust. For one thing, it is a standalone, individual program; basically no other components of Firefox are used in this way. Firefox itself launches many processes as a means of sandboxing and insulating against crashes, however these processes all talk to one another and have access to the same code base.

The crash reporter has a very unique requirement: it must use as little as possible of the Firefox code base, ideally none! We don't want it to rely on code which may be buggy and cause the reporter itself to crash. Using a completely independent implementation ensures that when a main process crash does occur, the cause of that crash won't affect the reporter's functionality as well.

The crash reporter also necessarily has a GUI. This alone may not separate it from other Firefox components, but we can't leverage any of the cross-platform rendering goodness that Firefox provides! So we need to implement a cross-platform GUI independent of Firefox as well. You might think we could reach for an existing cross-platform GUI crate, however we have a few reasons not to do so.

• We want to minimize the use of external code: to improve crash reporter reliability (which is paramount), we want it to be as simple and auditable as possible.
• Firefox vendors all dependencies in-tree, so we are hesitant to bring in large dependencies (GUI libraries are likely pretty sizable).
• There are only a few third-party crates that provide a native OS look and feel (or actually use native GUI APIs): it's desirable for the crash reporter to have a native feel to be familiar to users and take advantage of accessibility features.

So all of this is to say that third-party cross-platform GUI libraries aren't a favorable option.

These requirements significantly narrow the approach that can be used.

Building a GUI View Abstraction

In order to make the crash reporter more maintainable (and make it easier to add new features in the future), we want to have as minimal and generic platform-specific code as possible. We can achieve this by using a simple UI model that can be converted into native GUI code for each platform. Each UI implementation will need to provide two methods (over arbitrary platform-specific &self data):

/// Run a UI loop, displaying all windows of the application until it terminates.
fn run_loop(&self, app: model::Application)

/// Invoke a function asynchronously on the UI loop thread.
fn invoke(&self, f: model::InvokeFn)

The run_loop function is pretty self-explanatory: the UI implementation takes an Application model (which we'll discuss shortly) and runs the application, blocking until the application is complete. Conveniently, our target platforms generally have similar assumptions around threading: the UI runs in a single thread and typically runs an event loop which blocks on new events until an event signaling the end of the application is received.

There are some cases where we'll need to run a function on the UI thread asynchronously (like displaying a window, updating a text field, etc). Since run_loop blocks, we need the invoke method to define how to do this. This threading model will make it easy to use the platform GUI frameworks: everything calling native functions will occur on a single thread (the main thread in fact) for the duration of the program.

This is a good time to be a bit more specific about exactly what each UI implementation will look like. We'll discuss pain points for each later on. There are 4 UI implementations:

• A Windows implementation using the Win32 API.
• A macOS implementation using Cocoa (AppKit and Foundation frameworks).
• A Linux implementation using GTK+ 3 (the "+" has since been dropped in GTK 4, so henceforth I'll refer to it as "GTK"). Linux doesn't provide its own GUI primitives, and we already ship GTK with Firefox on Linux to make a modern-feeling GUI, so we can use it for the crash reporter, too. Note that some platforms that aren't directly supported by Mozilla (like BSDs) use the GTK implementation as well.
• A testing implementation which will allow tests to hook into a virtual UI and poke things (to simulate interactions and read state).

One last detail before we dive in: the crash reporter (at least right now) has a pretty simple GUI. Because of this, an explicit non-goal of the development was to create a separate Rust GUI crate. We wanted to create just enough of an abstraction to cover the cases we needed in the crash reporter. If we need more controls in the future, we can add them to the abstraction, but we avoided spending extra cycles to fill out every GUI use case.

Likewise, we tried to avoid unnecessary development by allowing some tolerance for hacks and built-in edge cases. For example, our model defines a Button as an element which contains an arbitrary element, but actually supporting that with Win32 or AppKit would have required a lot of custom code, so we special case on a Button containing a Label (which is all we need right now, and an easy primitive available to us). I'm happy to say there aren't really many special cases like that at all, but we are comfortable with the few that were needed.

The UI Model

Our model is a declarative structuring of concepts mostly present in GTK. Since GTK is a mature library with proven high-level UI concepts, this made it appropriate for our abstraction and made the GTK implementation pretty simple. For instance, the simplest way that GTK does layout (using container GUI elements and per-element margins/alignments) is good enough for our GUI, so we use similar definitions in the model. Notably, this "simple" layout definition is actually somewhat high-level and complicates the macOS and Windows implementations a bit (but this tradeoff is worth the ease of creating UI models).

The top-level type of our UI model is Application. This is pretty simple: we define an Application as a set of top-level Windows (though our application only has one) and whether the current locale is right-to-left. We inspect Firefox resources to use the same locale that Firefox would, so we don't rely on the native GUI's locale settings.

As you might expect, each Window contains a single root element. The rest of the model is made up of a handful of typical container and primitive GUI elements:

The crash reporter only needs 8 types of GUI elements! And really, Progress is used as a spinner rather than indicating any real progress as of right now, so it's not strictly necessary (but nice to show).

Rust does not explicitly support the object-oriented concept of inheritance, so you might be wondering how each GUI element "extends" Element. The relationship represented in the picture is somewhat abstract; the implemented Element looks like:

pub struct Element {
    pub style: ElementStyle,
    pub element_type: ElementType
}

where ElementStyle contains all the common properties of elements (alignment, size, margin, visibility, and enabled state), and ElementType is an enum containing each of the specific GUI elements as variants.

Building the Model

The model elements are all intended to be consumed by the UI implementations; as such, almost all of the fields have public visibility. However, as a means of having a separate interface for building elements, we define an ElementBuilder<T> type. This type has methods that maintain assertions and provide convenience setters. For instance, many methods accept parameters that are impl Into<MemberType>, some methods like margin() set multiple values (but you can be more specific with margin_top()), etc.

There is a general impl<T> ElementBuilder<T> which provides setters for the various ElementStyle properties, and then each specific element type can also provide their own impl ElementBuilder<SpecificElement> with additional properties unique to the element type.

We combine ElementBuilder<T> with the final piece of the puzzle: a ui! macro. This macro allows us to write our UI in a declarative manner. For example, it allows us to write:

let details_window = ui! {
    Window title("Crash Details") visible(show_details) modal(true) hsize(600) vsize(400)
         halign(Alignment::Fill) valign(Alignment::Fill)
    {
         VBox margin(10) spacing(10) halign(Alignment::Fill) valign(Alignment::Fill) {
                Scroll halign(Alignment::Fill) valign(Alignment::Fill) {
                        TextBox content(details) halign(Alignment::Fill) valign(Alignment::Fill)
                },
                Button halign(Alignment::End) on_click(move || *show_details.borrow_mut() = false)
             {
                 Label text("Ok")
             }
         }
     }
};

The implementation of ui! is fairly simple. The first identifier provides the element type and an ElementBuilder<T> is created. After that, the remaining method-call-like syntax forms are called on the builder (which is mutable).

Optionally, a final set of curly braces indicate that the element has children. In that case, the macro is recursively called to create them, and add_child is called on the builder with the result (so we just need to make sure a builder has an add_child method). Ultimately the syntax transformation is pretty simple, but I believe that this macro is a little bit more than just syntax sugar: it makes reading and editing the UI a fair bit clearer, since the hierarchy of elements is represented in the syntax. Unfortunately a downside is that there's no way to support automatic formatting of such macro DSLs, so developers will need to maintain a sane formatting.

So now we have a model defined and a declarative way of building it. But we haven't discussed any dynamic runtime behaviors here. In the above example, we see an on_click handler being set on a Button. We also see things like the Window's visible property being set to a show_details value which is changed when on_click is pressed. We hook into this declarative UI to change or react to events at runtime using a set of simple data binding primitives with which UI implementations can interact.

Data Binding

There are a few types which we use to declare dynamic (runtime-changeable) values. In our UI, we needed to support a few different behaviors:

• triggering events, i.e., what happens when a button is clicked,
• synchronized values which will mirror and notify of changes to all clones, and
• on-demand values which can be queried for the current value.

As a means of convenience, we created a Property type which encompasses the value-oriented fields as well. A Property<T> can be set to either a static value (T), a synchronized value (Synchronized<T>), or an on-demand value (OnDemand<T>). It supports an impl From for each of these, so that builder methods can look like fn my_method(&mut self, value: impl Into<Property<T>>) allowing any supported value to be passed in a UI declaration.

We won't discuss the implementation in depth (it's what you'd expect), but it's worth noting that these are all Clone to easily share the data bindings: they use Rc (we don't need thread safety) and RefCell as necessary to access callbacks.

In the example from the last section, show_details is a Synchronized<bool> value. When it changes, the UI implementations change the associated window visibility. The Button on_click callback sets the synchronized value to false, hiding the window (note that the details window used in this example is never closed, it is just shown and hidden).

Threading and Thread Safety

The careful reader might remember that we discussed how our threading model involves interacting with the UI implementations only on the main thread. This includes updating the data bindings, since the UI implementations might have registered callbacks on them! While we could run everything in the main thread, it's generally a much better experience to do as much off of the UI thread as possible, even if we don't do much that's blocking (though we will be blocking when we send crash reports). We want our business logic to default to being off of the main thread so that the UI doesn't ever freeze. We can guarantee this with some careful design.

The simplest way to guarantee this behavior is to put all of the business logic in one (non-Clone, non-Sync) type (let's call it Logic) and construct the UI and UI state (like Property values) in another type (let's call it UI). We can then move the Logic value into a separate thread to guarantee that UI can't interact with Logic directly, and vice versa. Of course we do need to communicate sometimes! But we want to ensure that this communication will always be delegated to the thread which owns the values (rather than the values directly interacting with each other).

We can accomplish this by creating an enqueuing function for each type and storing that in the opposite type. Such a function will be passed boxed functions to run on the owning thread that get a reference to the owned type (e.g., Box<dyn FnOnce(&T) + Send + 'static>). This is simple to create: for the UI thread, it is just the UI implementation's invoke method which we briefly discussed previously. The Logic thread does nothing but run a loop which will get these functions and run them on the owned value (we just enqueue and pass them using an mpsc::channel). Now each type can asynchronously call methods on the other with the guarantee that they'll be run on the correct thread.

Localization

One nice benefit of this rewrite is that we could bring the localization of the crash reporter up to speed with our modern tooling. In almost every other part of Firefox, we use fluent to handle localization. Using fluent in the crash reporter makes the experience of localizers more uniform and predictable; they do not need to understand more than one localization system (the crash reporter was one of the last holdouts of the old system). It was very easy to use in the new code, with just a bit of extra code to extract the localization files from the Firefox installation when the crash reporter is run. In the worst case scenario where we can't find or access these files, we have the en-US definitions directly bundled in the crash reporter binary.

The UI Implementations

We won't go into much detail about the implementations, but it's worth talking about each a bit.

Linux (GTK)

The GTK implementation is probably the most straightforward and succinct. We use bindgen to generate Rust bindings to the GTK functions we need (avoiding vendoring any external crates). Then we simply call all of the corresponding GTK functions to set up the GTK widgets as described in the model (remember, the model was made to mirror some of the GTK concepts).

Since GTK is somewhat modern and meant to be written by humans (not automated tools like some of the other platforms), there weren't really any pain points or unusual behaviors that needed to be addressed.

We have a handful of nice features to improve memory safety and correctness. A set of traits makes it easy to attach owned data to GObjects (ensuring data remains valid and is properly dropped when the GObject is destroyed), and a few macros set up the glue code between GTK signals and our data binding types.

Windows (Win32)

The Windows implementation may have been the most difficult to write, since Win32 GUIs are very rarely written nowadays and the API shows its age. We use the windows-sys crate to access bindings to the API (which was already vendored in the codebase for many other Windows API uses). This crate is generated directly from Windows function metadata (by Microsoft), but otherwise its bindings aren't terribly different from what bindgen might have produced (though they are likely a bit more accurate).

There were a number of hurdles to overcome. For one thing, the Win32 API doesn't provide any layout primitives, so the high-level layout concepts we use (which allow graceful resize/repositioning) had to be implemented manually. There's also quite a few extra API calls just to get to a GUI that looks somewhat decent (correct window colors, font smoothing, high DPI handling, etc). Even the default font ends up being a terrible looking bitmapped font rather than the more modern system default; we needed to manually retrieve the system default and set it as the font to use, which was a bit surprising!

We have a set of traits to facilitate creating custom window classes and managing associated window data of class instances. We also have wrapper types to properly manage the lifetimes of handles and perform type conversions (mainly String to null-terminated wide strings and back) as an extra layer of safety around the API.

macOS (Cocoa/AppKit)

The macOS implementation had its tricky parts, as overwhelmingly macOS GUIs are written with XCode and there's a lot of automated and generated portions (such as nibs). We again use bindgen to generate Rust bindings, this time for the Objective-C APIs in macOS framework headers.

Unlike Windows and GTK, you don't get keyboard shortcuts like Cmd-C, Cmd-Q, etc, for free if creating a GUI without e.g. XCode (which generates it for you as part of a new project template). To have these typical shortcuts that users expect, we needed to manually implement the application main menu (which is what governs keyboard shortcuts). We also had to handle runtime setup like creating Objective-C autorelease pools, bringing the window and application (which are separate concepts) to the foreground, etc. Even implementing invoke to call a function on the main thread had its nuances, since modal windows use a nested event loop which would not call queued functions under the default NSRunLoop mode.

We wrote some simple helper types and a macro to make it easy to implement, register, and create Objective-C classes from Rust code. We used this for creating delegate classes as well as subclassing some controls for the implementation (like NSButton); it made it easy to safely manage the memory of Rust values underlying the classes and correctly register class method selectors.

The Test UI

We'll discuss testing in the next section. Our testing UI is very simple. It doesn't create a GUI, but allows us to interact directly with the model. The ui! macro supports an extra piece of syntax when tests are enabled to optionally set a string identifier for each element. We use these strings in unit tests to access and interact with the UI. The data binding types also support a few additional methods in tests to easily manipulate values. This UI allows us to simulate button presses, field entry, etc, to ensure that other UI state changes as expected as well as simulating the system side effects.

Mocking and Testing

An important goal of our rewrite was to add tests to the crash reporter; our old code was sorely lacking them (in part because unit testing GUIs is notoriously difficult).

Mocking Everything

In the new code, we can mock the crash reporter regardless of whether we are running tests or not (though it is always mocked for tests). This is important because mocking allows us to (manually) run the GUI in various states to check that the GUI implementations are correct and render well. Our mocking not only mocks the inputs to the crash reporter (environment variables, command line parameters, etc), it also mocks all side-effectful std functions.

We accomplish this by having a std module in the crate, and using crate::std throughout the rest of the code. When mocking is disabled, crate::std is simply the same as ::std. But when it is enabled, a bunch of functions that we have written are used instead. These mock the filesystem, environment, launching external commands, and other side effects. Importantly, only the minimal amount to mock the existing functions is implemented, so that if e.g. some new functions from std::fs, std::net, etc. are used, the crate will fail to compile with mocking enabled (so that we don't miss any side effects). This might sound like a lot of effort, but you might be surprised at how little of std really needed to be mocked, and most implementations were pretty straightforward.

Now that we have our code using different mocked functions, we need to have a way of injecting the desired mock data (both in tests and in our normal mocked operation). For example, we have the ability to return some data when a File is read, but we need to be able to set that data differently for tests. Without going into too much detail, we accomplish this using a thread-local store of mock data. This way, we don't need to change any code to accommodate the mock data; we only need to make changes where we set and retrieve it. The programming language enthusiasts out there may recognize this as a form of dynamic scoping. The implementation allows our mock data to be set with code like

mock::builder()
    .set(
        crate::std::env::MockCurrentExe,
        "work_dir/crashreporter".into(),
    )
    .run(|| crash_reporter_main())

in tests, and

pub fn current_exe() -> std::io::Result {
    Ok(MockCurrentExe.get(|r| r.clone()))
}

in our crate::std::env implementation.

Testing

With our mocking setup and test UI, we are able to extensively test the behavior of the crash reporter. The "last mile" of this testing which we can't automate easily is whether each UI implementation faithfully represents the UI model. We manually test this with a mocked GUI for each platform.

Besides that, we are able to automatically test how arbitrary UI interactions cause the crash reporter to affect its own UI state and the environment (checking which programs are invoked and network connections are made, what happens if they fail, succeed, or timeout, etc). We also set up a mock filesystem and add assertions in various scenarios over the precise resulting filesystem state once the crash reporter completes. This greatly increases our confidence in the current behaviors and ensures that future changes will not alter them, which is of the utmost importance for such an essential component of our crash reporting pipeline.

The End Product

Of course we can't get away with writing all of this without a picture of the crash reporter! This is what it looks like on Linux using GTK. The other GUI implementations look the same but styled with a native look and feel.

Note that, for now, we wanted to keep it looking exactly the same as it previously did. So if you are unfortunate enough to see it, it shouldn't appear as if anything has changed!

With a new, cleaned up crash reporter, we can finally unblock a number of feature requests and bug reports, such as:

• detecting whether an installation is corrupt and telling the user to re-install Firefox,
• checking whether there is faulty memory hardware on the crashing system, and
• using the Firefox network stack for the first attempt at submitting crashes (which respects user network settings like proxies).

We are excited to iterate and improve further on crash reporter functionality. But ultimately it'd be wonderful if you never see or use it, and we are constantly working toward that goal!

The post Porting a cross-platform GUI application to Rust appeared first on Mozilla Hacks - the Web developer blog.

23 Apr 2024 7:08pm GMT

Highlights

• The team is in the early stages of adding wallpaper support! This is still very preliminary, but you can test what they've currently landed on Nightly:
  • Set browser.newtabpage.activity-stream.newtabWallpapers.enabled to true in about:config
  • Click on the "gear" icon in the top-right of the new tab page
  • Choose a wallpaper! Note that you get different options depending on whether or not you're using a light or dark theme.

Set a new look for new tabs!

• Among other things, Firefox 125 is going to ship with support for the Popover Web API
  • This should make it easier to create dropdown / popup menus without doing so much management with JavaScript. Here are some demos.
• Niklas added keyboard shortcuts for copying and downloading a screenshot in the Screenshots tool
  • The keyboard shortcuts work in the overlay and in the preview window
    • ctrl/cmd + c to copy the screenshot
    • ctrl/cmd + s to download/save the screenshot
• Nicolas fixed some performance issues in the Rules view in the Firefox DevTools Inspector tab when there are a lot of rules with pseudo elements. For example, with 1500 pseudo elements, what was taking 3 seconds on a developer machine now only takes ~30ms! These fixes are slated to go out in Firefox 126.
• jesup moved brotli decompression fully off of the main thread. This has shown major wins in the wild - we're seeing a 10% improvement for FirstContentfulPaint pretty much across the board, and a 10% improvement on LargestContentfulPaint at the 50% percentile, 20% at the 75% percentile and 95%(!) at the 95% percentile. Nice! This improvement is rolling out in Firefox 126.

Friends of the Firefox team

Resolved bugs (excluding employees)

Volunteers that fixed more than one bug

• Camille
• Magnus Melin [:mkmelin]
• Meera Murthy

New contributors (🌟 = first patch)

• 🌟 Camille added `text-align: start` to moz-message-bar so it doesn't inherit alignment from parent and updated moz-toggle to only show active state when hovering
• gravyant improved error message when sending session.new without capabilities
• 🌟 Harshit replaced `validateURL` with `URL.parse()`

Project Updates

Add-ons / Web Extensions

WebExtension APIs

• As part of the ongoing work related to improving cross-browser compatibility for Manifest Version 3 extensions:
  • The options_page manifest property is supported as an alias of options_page.ui - Bug 1816960
  • A new webRequestAuthProvider permission allows extensions to register webRequest.onAuthRequired blocking listeners (in addition to the webRequestBlocking permission, which is deprecated on Chrome but still supported by Firefox) - Bug 1820569
  • commands.onCommand listeners now receive details of the currently active tab - Bug 1843866
  • WebExtensions with a granted active tab permission can now call tabs.captureVisibleTab API method without any additional host permissions - Bug 1784920
  • MessageSender details received by runtime.onMessage/runtime.onConnect listeners include a new origin property - Bug 1787379

Developer Tools

DevTools

• Artem Manushenkov added a setting to disable the split console (#1731635)
• Yury added support for Wasm exception handling proposal in the Debugger (#1885589)
• Emilio fixed a rendering issue that could happen after exiting Responsive Design Mode (#1888242)
• Alex migrated the last DevTools JSMs to ESMs (#1789981, #1827382, #1888171)
• Nicolas improved performance of the Inspector when modifying a single rule, in a stylesheet with a lot of rules (#1888079, #1888081)
• Nicolas improved the Rules view by showing the color picker button on color functions using CSS variables in their definition (#1718894)
• Bomsy fixed a crash in the netmonitor (#1884571)
• Julian reverted the location of DevTools screenshots on OSX to match where Firefox screenshots are saved (#1845037)
• Nicolas added @property rules (enabled on Nightly by default) in the Style Editor sidebar (#1886392)

WebDriver BiDi

• New contributor: :gravyant improved the error message when the session.new command is used without a proper capabilities parameter (#1838152)
• Julian Descottes added support for the contexts argument of the network.addIntercept command which allows to restrict a network intercept to a set of top-level browsing contexts (#1882260)
• Sasha Borovova updated the storage.getCookies command to return third party cookies selectively, based on the value of the network.cookie.cookieBehavior and network.cookie.cookieBehavior.optInPartitioning preferences (#1879503)
• Sasha Borovova removed the ownership and sandbox parameters for the browsingContext.locateNodes command to align with a recent specification update (#1884935)
• Sasha Borovova updated the session.subscribe and session.unsubscribe commands to throw an error when the events or contexts parameters are empty arrays (#1887871)
• The team completed the Milestone 10 of the project (bug list), where we implemented 50% of the commands needed to completely support Puppeteer, with 75% of the Puppeteer unit-tests passing with WebDriver BiDi. For Milestone 11 (bug list), our focus remains to implement the remaining commands and features required to fully support Puppeteer (doc).

Lint, Docs and Workflow

• Enabling ESLint rules for requiring JSDocs just got easier.
  • The rules are now enabled by default.
  • Places where there are currently failures are explicitly disabled in the top-level .eslintrc.js file.
  • Please consider enabling these (and the valid-jsdoc ones) on your area, especially if you are using (or will be using) JSDoc document generation for source docs.
• eslint-plugin-mozilla now supports ESLint v9 APIs.
  • We still have some work to complete to fully support the new flat configuration.
• We're now rejecting the JSM based ChromeUtils.import across the tree via ESLint.

Migration Improvements

• Negin fixed an issue where the migration wizard title wasn't appearing when embedded in about:welcome
• Negin is working on fixing an issue on about:welcome where clicking on the browser/profile selector when the panel is already open doesn't close the panel.
• For the backup project, we've made some pretty big strides!
  • We closed out the measurement metabug, and have the measurements being collected by default starting in Firefox 126.
  • We're now staging backups for various resources
    • User preferences
    • The Places databases (history, bookmarks, favicons)
    • Credentials (passwords, certificates, payment methods, addresses)
    • A bunch of miscellaneous data stores
  • We added a very simple debug tool in Nightly at chrome://browser/content/backup/debug.html. You can test staging a backup at runtime with what we have so far using this tool. It may take several minutes depending on how big your databases are, and how intensely you're using the browser while the backup is underway.

Picture-in-Picture

• Thanks to :joe.scott.webster for submitting a patch that fixes PiP captions issues with several Yahoo sites (bug) and filing a follow-up ticket for AOL (bug).
  • Also thanks to Niklas Baumgardner (:niklas) for lending a hand!

Performance

• mconley got rid of some nsTerminator telemetry that was causing us to block shutting down the browser if a quit was requested very soon after a startup.

Screenshots (enabled by default in Nightly)

• Niklas made the save full page the primary action button
  • Thanks reusable components for moz-button-group!
• Niklas fixed an issue where dragging the scrollbar could drag a region
• Niklas fixed an issue where the window would sometimes scroll when focusing the download or copy button

Search and Navigation

• Firefox Suggest experience
  • Daisuke renamed the "Learn More about Firefox Suggest" menuitem to a more direct "Manage Firefox Suggest". Bug 1889820
  • Drew added new telemetry to measure in experiments the potential exposure of simulated results, depending on the typed search string. Bug 1881875
• SERP categorization telemetry
  • James, Stephanie and Karandeep have landed many fixes to the storage, logging and measurements.
• Search Config v2
  • Enabling new config in Nightly lowered the number of initialization errors for the Search Service
  • Mark fixed character-set handling. Bug 1890698
  • Mark added a new property covering the device type. Bug 1889910
  • Mandy sorted collections by engine identifier and property names, to make the config more easily navigable and diffs nicer and easier to maintain. Bug 1889247
  • Mandy updated the documentation. Bug 1889037
• Other fixes
  • Marco fixed a bug causing engagement on certain results to be registered both as engagement and abandonment. Bug 1888627
  • Dao has fixed alignment of "switch to tab" and "visit" chiclets. Bug 1886761

Storybook/Reusable Components

• Thanks to Camille Davis for fixing these two bugs!
  • Bug 1856717 - Set 'text-align: start' on ':host' in moz-message-bar.css
  • Bug 1883058 - moz-toggle should only show active state on hover
• Thanks to :nordzilla for fixing a race condition in menulist
• Thanks to :kcochrane for adding documentation to moz-page-nav and adding support for external and support links in moz-page-nav
• :hjones fixed the interactive storybook component examples on Windows
• :tgiles fixed the ./mach addstory command
• Work is underway to remove the window.ensureCustomElements function now that we can lazy load ES modules. See Bug 1803678 - Enable on-demand/lazy loading for ESModule based reusable components for more details
  • Big thanks to :arai for the work to make this possible! Bug 1803810 - Support synchronously loading ESMs into given global, in the same way as Services.scriptloader.loadSubScript

23 Apr 2024 5:32pm GMT