Yarn — 2019 and Beyond by Maël Nison | JSConf EU 2019


>>So you might have seen the talk from Kat
just before. It’s really interesting, and you should keep
it in mind while we make this one as there are a few things that might overlap a bit. First, who am I? My name is Maël. I used to work at Facebook as part of the
JavaScript foundation team. My job there was to en – do the infrastructure,
and I came to contribute to Yarn. As time passed, I started to take more and
more responsibilities on the project, and here I am today talking about the work we’ve
been doing for the V2 which is in two small part based on my observation after using Yarn
at scale for a few years. Something I like to remember when doing this
talk is why does Yarn exist in the first place? How did it get created? It all started with three companies – Facebook,
Google, and Expo – which felt something was missing in their infrastructure. They figured the existing solution didn’t
meet all their needs in terms of stability, user experience, and so they decided to try
something different, to try something new. A different solution with different architectural
choices, different trade-offs that would still be compatible with the ecosystem at large. That was a huge bet but we succeeded, and
that is how Yarn got created. Since then, we keep this philosophy, think
of Yarn as an a laboratory. Speaking of feels, before talking about what
Yarn 2 will bring to the table in 2019 and beyond, we must first go quickly over what
it contributed and more importantly what are the things they did right. It’s the important to know because that’s
how we can ensure that we won’t lose what makes our core values when we make the transition
from the v1 to the v2. This is an open question, by the way. I would like to hear what you think Yarn did
right. Please come and see me after the talk and
share the stories. Ever since Yarn got created, something we
deem very important was stability. We want JavaScript project to be stable. We want you to trust your project, and we
are ready to go the extra mile to achieve this goal. This is about teaching good practices and
letting you know when something is wrong before it can cause any article. We work to remove unsafe patterns from the
equation as much as possible. Our log files ensure that the install dependencies
are always the same, our install strategy, plug ‘n’ play, prevent your application breaking
at deploy time, and our offline mirror ensures that the network will never be a problem in
deployment. Speaking of the network, this is one thing
in particular we worked really well for us. At Facebook, we barely even noticed when the
registry was down. In fact, we often learned about it through
Twitter, really. Despite the offline mirror being one of our
less-known features, it might be one of the most important ones in my mind. You should definitely check it out. If you don’t, it’s fine, because it will soon
become unmissable, starting from the v2 where it will be enabled by default. Going back to the things we did right, work
spaces are high on the list. If you don’t know them pretty well, they allow
you to split your applications into sub projects that can live independently from each other
while style being able to cross-reference each other. Ing see them as a declarative Yarn link. For example, you can see here we’ve been able
to split our huge blob of dependencies into smaller pieces, one for the back end and one
for the front-end. Spaces came to the community, but as we noticed
how much they improve our own product, we decided to make it a core feature in Yarn. That was a great idea. Given the success they got, we will be double
ing down our efforts to make them first-class citizens, as you will soon see. Finally, the last core value that we definitely
should mention is the overall user experience. It doesn’t seem of, but it’s actually a big
reason why our users say they use Yarn. The small things, like the run key while not
being native, or the merge conflicts automatically being resolved. Those things aren’t the big shiny things you
typically that are on our side, but they are the comments that you use every day, the ones
that shape your work flow, and that is something we keep in mind when working on the v2. Okay, so now we have to talk a bit about the
elephant in the room: why are we even talking about a new major Yarn release today? About half of you use Yarn in your life with
great success, so how would a new major release affect you? Do we really have something significant to
bring to the table? If so, what? Something to realise is that Yarn got created
in 2016. Since then, a lot of things happened. For one, we released a lot of major features. Work spaces in 2017, plug ‘n’ play in 2018,
zip loading is coming in 2019, and we will go to that later. Our architecture needed a refresh to stay
up to date with all those new concepts that didn’t even exist when Yarn got created, and
to support the new work flow they unlock. Additionally, Yarn got initially created as
an npm clone. As such, we … values behaviours that won’t
mesh well with the user experience we provide which is something we really care about. For those technical reasons, we needed to
build something new, to build a new modern foundation more suited to what we had in mind. That’s not the only reason. The main reason in my mind is that we wanted
to give you more power. We wanted to make it easier for you to contribute
to Yarn. To write your own features, to fix what blocks
you, not to have to rely on us. After all, Yarn is a community project. We don’t want to work for you, we want to
work with you. So our architecture needed to reflect that,
and for this reason, we decided to make it very modular so that you would be able to
write your own plugins. In fact, most of the features in Yarn 2 are
actually plugins that got aggregated together in a single binary. In the same vein, we moved our codebase to
a more familiar stack-typescript, and we hoped that, by doing this, it will decrease the
time needed for you to understand the pieces, how they fit together in such a way that you
will be able to help us maintain this critical part of your infrastructure, and it’s scope
to … enough about the abstract talk, let’s delve a bit into what the next major version
of Yarn will concretely bringing to the table. The first thing you will notice is that we
revamped the output. What Yarn prints is how it communicates with
you, and we wanted to make it easier for you to figure out what happens and highlight the
most important information. This translates into important way: first,
we know print message codes that can be Googled for more detail on what a specific message
means, and, if if the in the case of errors, how to solve them. Secondly, we are now using colours with a
specific purpose to highlight the dynamic parts of our messages. It’s one of those very tiny things that I
mention that happened to matter a lot when you work with Yarn every day. By using the colours at the right place, we
aim to decrease the – you won’t have to fight to understand it. The v2 also comes with new comments. You might have heard about Yarn creating the
v1 which allowed you to download and run any package prefixed by the word “create”. We decided to expand it and build Yarn DLX. That very similar to its sibling and PX can
install packages from the registry. It allows you to run 1&1-off strips without
having to revert your changes after the fact. Let’s see a quick demonstration of it. So here, we are a JavaScript file which is
webpack configuration, and we will want to minify it. Usually, we would have to add the minimum
fire filter into our package at JSON and then remove it – but by using DLX, you just have
to run Yarn DLX and the tame of the file, and Yarn will download everything’s that is
needed and run it locally. But even better, since we are using plug ‘n’
play, it means that we don’t have to create … more, and DLX at the second invocation
will be just as fast as it can be because we don’t have to fetch the package again. If I run it again, it will be super fast. So that is for newcomers. We have other ones. We’ve also improved the existing ones. Various changes were made to make it easier
to work with mono repos or fix long-standing issues peeve. Yarn hub can upgrade a package across all
your at once. Yarn Link is able to properly resolve peer
dependencies even when you link an external package into your current product. Yarn Ad accept an interactive flag that be
instructs Yarn to ask you for advice in various situations. And it’s not only a toy. The interactive mode, for example, allows
you to quickly reuse a version of a package that another one of your work spaces is already
using. You don’t need to grab into your packager
JSON any more to find that out. Speaking about – we are introducing constraints. Wouldn’t it be nice if you could just check
that your project obeys some rules? Like linting, some kind of ES lint, but for
package.json. We thought so too. One of our newest conditions are constraints
which are basically lints rules and written in prologue. Prologue makes them extremely small and powerful. In the example, you can see our work spaces
forbids Babel listed as an a regular dependency. In fact, with only two more lines, we could
extend this to automatically move the dependency within the dev dependency field. Constraints are very powerful, and they have
many usages. You can see some of the screen, and we would
provide some recipes in our documentation to get started, but you will quickly see that
it is super easy to write new ones that are directly adapted to your use case. We can see them in practice a bit. So here, I have project with two work spaces:
my front-end and my back-end. Both my front-end and back-end are depending
on low dash except both are TypeScript and should depend on – it would be nice if there
was a way to detect all of those problems and fix them as best as we can. Yarn as a newcomer, Yarn constraints check,
when Yarn constraints check is called, it will validate the project against the defined
rules, so, in this case, you can see that it has been able to detect that back-end and
front-end must depend on TS … and they depend on conflicting versions of lodash. If I run Yarn constraints fix, Yarn will automatically
try to fix all the problems that it detects and can fix. In this case, for example, you can see that
it is able to add TSLIB to the dependencies. It hasn’t been able to correct the mismatch
in lodash because it’s ambiguous. It doesn’t know which one you want to use. In order to fix those kind of issues, you
have the good old Yarn Add. If you run that with interactive mode, Yarn
will ask you which version of Yarn you want to use amongst multiple possible ones. In this situation, it is smart enough to tell
you that lodash is already used in two work spaces. Which one do you want to use? By using this feature, you don’t have to think
about how should I fix the problem? You just have to validate and Yarn will automatically
continue and do its stuff. You will see that all the … have been properly
fixed and everything has been applied. Now, something else we can do with constraint,
because of how powerful they are, is using them as a query. By using Yarn constraints Query, you get to
ask Yarn what do you know about my project? So, for example, in this case, by using one
simple query, I’m able to ask Yarn to tell me all the dependencies and all the ranges
in all of my work spaces. But I want – I can be more precise if I want
to. I have to specify one of the parameters and
Yarn will be able to filter those. That is the power of the constraints. Another feature that we are working on, and
that is maybe the biggest feature that we will ship, is zero install. Zero install is not actually a feeling, it’s
more of a concept, and it’s more aimed for applications than libraries, even though they
work for both. The idea is never run Yarn Install again. That might remind you of something! The idea behind this is that if a code needs
to run, this code will fail, even though we try our best to make a package manager that
never fails, from time to time, we’re not free of making a bug, or making a mistake,
and is it will crash at some point, or if it is not us, maybe it’s – maybe your file
system will be run – that actually happened to me before. So, in order for something to work every time,
it doesn’t – it must not run. So, with the zero install, the concept is
simple. You just add all your Yarn files into the
repository. And then you don’t need to run Yarn install
because everything will be there. I know what you think that committing the
directory bad. With plug ‘n’ play, we don’t have a directory
any more and plug ‘n’ play is the default in v2. The problem can committing non-models was
that non-models contain 20,000 text files. You may remember that I mentioned zip loading
before. With Yarn 2, instead of generating an unpacked
cache, we just download each package that you use, and we store them within your project,
or elsewhere if you don’t want to. Which means that, for each package that you
use, you will have exactly one archive. And that is peanuts for it to support. We discussed it with GitHub before breaking
their infra, and they told us it would be fine. So what does zero install bring to the table? Kat mentioned it improves the developer experience
by a large factor. We don’t have to run Yarn style between git
pull and checkout. Peer reviewers know which packet has been
– they look at the file’s change and see what has been added into the project. It is faster, simpler, and more stable for
deployments, because again you don’t have to run Yarn at all, so there is no risk of
Yarn crashing in production. Additionally, you will always have exactly
the same state on production as you have on development, because that is the guarantee
we provide. If it works locally, it will work in production. Additionally, it allows you to skip the CI
configuration step. So, in this case, we can see a quick demonstration
of zero install, so you can see that we have a small project that is using webpack 3. We would like to update to webpack 4, so we
create a new branch, and here, we will just run Yarn add webpack four. Yarn is now running, downloading be the dependencies
as it always did. If we do after that git status, we will see
something very interesting. But first, yes, you can see that webpack got
updated correctly, of course. And now if we run GIT status, you can see
something interesting. Instead of having … we have as I mentioned
one zip for each package. If we commit all this, then we have everything
we need in order to run our applications. Let’s say that suddenly a project manager
comes into your room and says we need to fix this problem on master, and it needs to be
done now, so here you need to give a switch quickly into the webpack 3 branch, make your
child, and commit it. But just after switching, you don’t have to
run Yarn install because everything is like it should be. Your project is always in a good state. Once you’re done, you just have to switch
back on your webpack 4 and everything will work just the same. By having this work flow and the simplifying
context switches, it makes you more productive because you can work on multiple features
at any point in time. In fact, it’s super useful for me and a Yarn
maintainer, because I can quickly check out the pulls submitted to Yarn and fix them myself,
something that was before very painful because I had to be careful in which state my project
– it’s now super easy, in fact, Yarn to itself is using zero install, so, for example, here,
you can find that I’m – so it’s a bit slower than it would be if you don’t have those tar
balls inside your repository but it’s faster than it would be if you were to clone your
repository and then run Yarn install. In the end, there’s a kind of balance here. And, yes, there is a balance here. So the clone is about to finish. And starting from now, you can use any comment
that you would typically use. For example, here, I can run the unit test
for Yarn. I don’t have to run Yarn install. Everything is already there. It makes it much easier for our contributors
to jump into the project. They just have to clone and run the counter. They don’t think more about it. Let’s recap a bit. We have an improved login. We have new comments. We have improved comments. We have added packages and linting. We have removed the need to run Yarn Install. That has to be worth a major bump, right? That is the tip of the iceberg. This is only a 30-minute talk, and there is
only so much that we can discuss here. Fortunately, we have a more comprehensive
change log on our GitHub repository. I encourage you to look at it to see what
is coming. The v2 is flagged as experimental but it’s
starting to become stable enough for your future projects. The official release is still to be determined
but we are thinking somewhere in the second half of 2019 once we’ve ensured the transition
path is as smooth as possible. A big thanks should go to our contributors,
of course, because we have had the pleasure to welcome various you in people who simple
stepped into the project and improved what the feature they liked. I said Yarn is a community project at the
beginning of my talk. And I really meant it. We are always looking for new contributors. More than that, we are ready to go the extra
mile and help you get started. Yarn is a huge project with a huge reach,
a huge impact but we still aim to make it approachable by anyone. Plus we have cool T-shirts for the v2 if you
contribute to the project, of course! So I will now leave you. I hope you enjoyed this talk. Please come ask your questions when you see
me in the hall. I will be thrilled to share more about the
work we do. Thank you, and have a good evening. [Cheering and applause].

Tags:, ,
4 Comments

Add a Comment

Your email address will not be published. Required fields are marked *