Kyle Simpson - Choose your own JS adventure
Fronteers 2014 | Amsterdam, October 10, 2014
Yeah, yeah, ES6 is awesome. Everyone knows that. We're all excited about not having to write the word 'function' anymore. But, isn't it about time we stop obsessing so much about what TC39 puts into the official language spec?
Let's talk about making and using tools (built in JS, of course!) to customize the language itself to your own liking. We'll talk about macros, code parsers and transpilers, and other fun tricks to put the control of the language back in your fingers.
(jake archibald) --a big Fronteers welcome to Kyle Simpson.
Appreciate it very much, Jake.
They gave us the whole stage to use, but none of the other speakers have done this, so I'm going to take a little bit of artistic liberty and I'm just going to start out my talk over here.
And I think I'll make myself comfortable so I'll go ahead and kick off my shoes, just like Jake did and hi, Marten.
Would you take my shoes for me? That would be awesome.
Thank you very much.
I appreciate it.
A big hand for Marten, please.
In all seriousness, the hospitality of this conference has been fantastic, just top notch and I do a lot of conference speaking, so I can say with authority that they've done a really great job.
So thank you all very much for having me here.
So we're going to start out just a little bit informally.
I just would like to know, not that I can actually see you because of the bright lights, but I am just curious.
How many of you have ever had a piece of code that you wrote that broke and you weren't sure why it was broken? I'm guessing by the laughs and by the noises that most of you raised your hand.
Here's a more profound question.
How many of you have written a piece of code and it worked and you still didn't know why it worked? Yeah, I thought so.
So I actually-- Sartre, if I'm pronouncing that correctly, he's famous for the "hell is other people" and then we've rephrase that in the development industry to, hell is other people's code.
But I'll tell you that hell is not understanding my own code.
And what we're going to get into today is this idea that source code is really for the developer and it's not for the computer and I'm going to come back to that theme several times.
So just to kind of set some expectation, the running theme for this conference thus far seems to at least, to me have been sort of pushing back your boundaries and saying you were using that technology or that practice incorrectly and I'm going to give you a different way of doing that.
And in that same theme, this talk will kind of challenge some of those perspectives that you may have had.
So on the first day, we saw, of course, CSS be taken down and this whole idea that there is best practices was sort of challenged and then we were told that we shouldn't even be writing markup anymore.
We should let tools to all that for us.
And that's been a running theme through many of these talks and I'm excited about that because I feel in its own little way that this talk can sort of fit into that.
So if any of you remember the old school choose your own adventure books, we're going to have that sort of style here.
And so I'm known as get a fine line.
If you have any thoughts or feedback that you'd like to provide, please do feel free to reach out.
You can also buy them because they're being published through Riley.
But I don't bring that up just to sell books, I actually bring it up because it brings up an important mindset that I want to make sure everybody understands when we get into this talk.
This is sort of a meta talk.
So if you've struggled with writing code and you've struggled with the language, the first step, of course, is to understand the language better and hopefully, these books can do that.
You really do need to understand the language in a very deep level, but this next step is sort of the step that I'm proposing today.
So we're going to jump right in with invoking the whole choose your own adventure.
You're Branden Iche and you're employed at Netscape in May of 1995 and your boss asks you to design a new language from scratch for the browser, but he gives you only 10 days to complete it.
So choose your next adventure.
To agree and give it your best shot, turn to page 39.
To laugh and walk away, turn to page 262.
And many of us probably wish that we'd turn to page 262, but we'll go to page 39.
And then the next adventure then we get to choose.
For example, this empty array is equal to the negation of an empty array.
That's just too unbearable for you to go on.
So choose your next adventure.
So enough of this silliness out the way, but I'm going to invoke right from the very beginning this idea that we need better tools.
Whenever we run across problems as developers, we oftentimes will blame the tool set that has been presented to us.
For instance, we might go in the language.
We might blame our editor.
We might blame some sort of process.
And rather than turning an eye towards blame, I'd rather this talk be turning an eye towards, let's stop focusing so much on the standards and the processes that we already have and turn our attention and our focus on creating an inspiring better tools.
So the very first thing I'm going to do is invoke one of those timeless, religious debates, this idea of spaces versus tabs.
And yes, this slide does, in fact, have a space on one side and a tab on the other.
You'll have to download the slides to figure out which one.
But everybody has a different opinion on these things and there really is not one correct answer.
Regardless of what a particular group may tell you, there really is not one particular answer, but teams will fight over this and developers will fight over this and it's one of the longest running battles in our development industry is this idea of spaces versus tabs.
And there's a strong contingent especially in the node world that will tell you semicolons are bad and you should not put them in because they make your code really ugly and it's too much the type to hit Shift-Semicolon or whatever.
So we want to remove our semicolons from our code and I would be in that camp.
Other people would say, no, no, no, we need those.
So what if, and this is the question I want to pose, what if instead of us arguing about these sorts of things, instead of us raging these debates and forcing developers to believe differently than the rest of the team to conform, what if we could just agree to not have to agree.
What if we didn't have to agree on these things.
What if that's something that a tool could solve.
So I am experimenting with-- and this is actually a much longer process than I thought because it turns out that we need a lot more information out of our code parsers, but I'm working on this tool called esre and this tool is designed specifically to take coding styles that you might say this is our standard for our team, but somebody else on the team they have a different set of standards for their styles.
And rather than having a linting tool that's simply complained at you, this tool will just fix it.
Isn't that amazing concept that rather than complaining at us as developers and putting a barrier to our workflow, we could create tools that just fix things.
So it's a configurable two-way formatting tool.
Instead of you having to change what you've done, the tool will automatically do it for you.
But this is not just a talk about code styles, it actually goes much deeper.
Are anybody aware in the audience the name of this movie? It's an old movie, obviously, the black and white days.
It's called "12 Angry Men."
And actually I love to think of this particular scene in this particular movie because I love to think of it as what happens inside of the TC39 Committee meetings although, obviously, it's not that there's certainly much more inclusive.
But I tweeted out this joke a while back and Allen Wirfs-Brock who's the maintainer of the ECMAscript spec, he wrote back to me and he said, I want to make sure that I claim that I'm the guy in the snazzy stripe suit.
So that's Allen Wirfs-Brock in our picture.
And let me be clear.
I have an immense amount of respect for what they're doing on the standards committee, what they're doing with our language pushing it forward.
So I'm very excited about the future of the language and I'm very glad that those people are shepherding the future of the language, but it's also very opaque to many of us.
And even though we can provide feedback on Twitter and we can even provide feedback through the discussion lists and things like that, there's a significant barrier there.
Because every time I would post an email to the list, I would get smacked down with, oh, you're using the wrong terms with things.
And so it is not necessarily as easy for all of us to participate as we might ideally hope.
And so part of this talk is to sort of liberate us from these processes and these standards and these tools.
They have to serve an enormous set of developers, an enormous audience with lots of different competing interests.
But what if you didn't have to settle for that compromise.
What if you could choose to customize this language? I go back to what I said at the beginning.
Source code is not for the computer.
Source code is for the developer.
Your computer only cares about ones and zeroes and there's a nearly infinite number of programs that can produce the same sequence of ones and zeros.
So your task as a developer is not to argue about spaces versus tabs, your job as a developer is to figure out the language and the tool set and the process that works most naturally for you, freeing you from all of those barriers so that you could write the best code possible.
And that's a somewhat counterclaim because many people think the process standardization is the only way to move forward.
We've heard talks that have suggested that we have to standardize on everything and there has to be a convention and a configuration for everything, but I'm pushing back on that and saying, perhaps that's optimizing for what I call a local maximum and it misses the big picture, the big picture that we as developers have individual likes and styles and things that we prefer.
So I'll first start by talking about a project called sweet.js.
And it caught on and it's actually gotten so much popularity that it's been adopted officially by the Mozilla family and they're shepherding this project and continuing to fund work on this project and there's a very, very strong chance that in a near coming version of language, perhaps ES7, maybe ES8, in that sort of time frame, we're going to see macros land a language and they're almost certainly going to land to look very similar to this.
Now, what happened was a group of people said, let's research and experiment with a set of features that we want to add that are not standard yet.
Let's figure those things out and that should sound familiar to you if you know anything about CoffeeScript.
So we'll write CoffeeScript.
And CoffeeScript, although I don't personally like it, is a fantastic thing for our community.
In fact, many of the greatest things are coming in ES6 came directly because of the experimentation that happened in CoffeeScript.
Another example, to give props to the Microsoft world, TypeScript is another perfect example of a tool that's been created in such a way to experiment with a set of things that are not yet standard, but hopefully could be one day.
And they're already on record as saying some of the things that TypeScript has experimented with are likely things that will make their way into the language in some way, shape, or form.
So turning our attention to this sweet.js,
let me just give you a very brief look at the sorts of things that you might be able to do with macros.
If you know about macros in other languages, then this will be fairly familiar to you.
So here's some ugly code that I'm doing to do a very common developer task, which is I have an x variable, a foo variable, and a bar variable and I need to swap those two.
I need to swap an x and a y in a general sense.
And we know that we're going to have to have some sort of temporary variable to do that, so you use syntactic tricks, which are coming.
So swap is actually a very basic but useful example of how you can define a macro.
So the beginning part is the definition of the macro from lines 1 through 9 and then down at the bottom, the important part, is that all we have to know is that we call this thing that looks like a function, but it's not really a function.
It's a language-level construct that's been added as a macro.
And there's all kinds of other tricks that you can pull to create new operators a new syntactic tricks to solve these sorts of problems.
Another example, this one is even more common, at least in my code and maybe you've run into these sorts of problems.
You have a nested object with several properties nested deep and you need to check for the third or fourth level deep, but you're going to have to check all levels of that because if you check the first level and it's not in there and you check the second level and it's not there, it can throw some sort of runtime error in your code.
And that's annoying to some people that come from other languages, but this is something that we can fix with macros.
So this is actually directly from their site and the syntax is a little complex, I'll completely admit, but the usage of this is that I can say nullity x.y.z
and it will automatically transform internally that code to run x ampersand, ampersand x.y, and x.y.z.
So it will do those tests in a safe way.
So it gets all of those details that have been frustrating you about that particular part of the language and it shoves them away.
It makes you more productive developer and I think that's actually a very positive thing for the future of the language and I think it's something that we can begin to use, not only to solve our problems, to be more pragmatic about our development, but also to experiment with the future versions of the language.
Think about it.
But it's more than just macro because macros are powerful and important, but they are sometimes limited.
So I'm thinking about something even bigger than this.
Let me start with an example.
And so this a variable out on line 6, it's available even though you probably intended as a developer for it to only be available inside the block and this has created untold numbers of bugs and programs where people accidentally access things.
So block scoping is an important and powerful technique, but unfortunately, it's just now landing in the language.
The good news is that we're getting a let keyword as of ES6 and that will create a variable declaration that will be block scoped.
So this new let keyword, it's a big win for ES6.
And it's going to allow us to enforce the things that we've been stylistically doing.
For example, any time you've done for var i equals, when you do an i declaration inside of your for loop, you're signaling to yourself and others, your future self and your other team members that you attend i to only be used for loop, but it doesn't belong only to the for loop if use the var and it does if you use the let keyword.
In fact, it solves many troubling issues about closures and so forth.
So block scoping is a huge win for the language.
I'm a big fan of it.
Just as a side, by the way, some people are out there saying that you're going to replace all vars with let, so let is the new var.
That's incredibly stupid advice.
Please don't follow that.
Do not just do a global find a replace on your code and write all new vars with lets.
They're going to be both useful as far as I'm concerned in your code.
You're going to use some vars and you're going to use some lets, but I encourage you to read up on some of that and it would be good if somebody had written a book about scoping.
So let's move on.
There are some problems with block scoping though.
In particular, with this particular form of block scoping, this is what I would call the implicit let declaration and essentially what it does is it hijacks an existing block.
There's a block there that's an if statement and it's hijacking it to give it some block scoping.
So let's look at slightly more realistic-- obviously, this is still stupid code, but it's slightly more realistic because it has more lines of code in it.
And it's actually going to show what I think is likely going to happen with let declarations is that they're going to get sprinkled throughout.
So you see on line 2, you see a let declaration, on line 4, you see another one.
And you're just going to subtly not realize until you've been bitten by this enough times that let declarations don't work quite like vars.
They don't hoist to the top of the block, so if you were to try to use a before line 4, you're going to get breakage and there's some weird things.
So it creates a lot of mental tax here.
And it's more than just that.
There's a refactoring tax.
So prior to this when we had declarations, no matter where we put them, they always belonged to the entire function scope, which meant we could move if statements all around and it wasn't a big deal.
We could sort of play loose and fast with the rules of scoping.
Now, that wasn't good code.
I'm not suggesting that that was a good thing, but this sort of code is a reality in our industry.
I've written this code, you've written this code, everybody's done this.
And if you replace them all with lets and then you start willy-nilly saying, well, I'll just take that if statement and I'll move it outside of the block, you're going to get refactoring hazards because you're going have to remember what needs to moved.
So you do need to be careful with the let statement and especially in its implicit form.
I would prefer an explicit form of block scoping and it turns out there is a form of the let declaration that was pioneered over a decade ago in Firefox and it's called the let block or otherwise known as the let statement.
You'll notice there line 3, I'm declaring an explicit block, a let a and I can do multiple variables in there.
That's an explicit block that does nothing except create scope.
Explicit is usually preferred by developers and this is one of those cases where I think explicit is better than implicit.
Some people hate the extra indentation, but I think that it reduces the mental tax, it reduces the refactoring hazard.
But there's a problem because this syntax was unfortunately rejected by ES6.
It is, in my opinion, a superior syntax, but it was rejected.
So what do we do? Do we just say, well, I guess we're just relegated to writing crappier code.
That's sort of the inspiration for this talk is that particular example and we'll come back to that in just a minute.
But let's take a step back and ask what exactly is it that I'm suggesting.
Am I suggesting democracy or anarchy? The answer is, yes, I'm suggesting all of these things.
I'm suggesting that it's OK for you to take matters into your own hands.
It's absolutely OK for you to decide what is a tool set that's going to work better for me.
How is it going to make me more productive as a developer.
Now, I know in the back of your minds there's going to be a question about collaboration and I promise we'll get back to that, so hold on to that question.
I need to create a tool or a set of tools to fix them, so FoilScript is essentially my unofficial collection of complaints about the language.
Sometimes there are loose parser modes, but in many cases, our current set of tools would fall down.
And the initial reaction might be say, well, let's use regular expressions.
Those are what I call the difficult literals.
If we had a tool that could sort of figure out where those were in our code and pre-identify them and pull them out, the language becomes significantly easier to reason about even with regular expressions.
It pre-identifies them leaving you with a much simpler set of strings of code that you could run regular expressions against to say find various different things you want to do.
And I'm using that particular tool in a tool that I'll talk about in just a moment.
So that's one step.
That's the first step towards building these sorts of crazy tools is that we need help and this tool might help.
Let's go back to our block scoping example for just a moment.
This particular block, I would prefer to write code like this because it's more explicit, but it is it going to run in the browser because it's not ES6 standard.
Maybe they'll come back and standardize it later, but right now it's not standard.
So what am I going to do? I'm going to write a tool I wrote a tool called let-er that takes that particular code and by default, it will now transpile it to the code you see there.
You see, all I did was just move the characters around a little bit.
It's actually not a significant change, but the second snippet is actually ES6 standard code.
So it's a very simple transpilation of our code from something nonstandard to something standard.
And it allows meet our author code in something that I think is more reasonable, it has less mental tax, and yet still target my code.
So we're not even talking about significant rewrites like something like CoffeeScript.
It can be very simple things like this.
Now that's the ES6 mode and my preferred option would be that you would use that and then, of course, use the rest of ES6 syntax.
You're going to use some generalized transpiler like Google Traceur or some of the others out there and you're going to target ES5 code and they've got a variety of ways that they do that.
I'll show you Traceur in a moment.
But let's say you want to use let-er totally standalone and you're not going to do anything except block scoping.
There is an ES3 mode and it produces this crazy hacky code, which is forcing a tri catch and the catch block is actually block scope.
That's the way it was defined way back in ES3.
So yeah, we don't want to write pack hacky code like that.
It's not a great solution.
Use one of those other tools if you prefer, but this is a standalone way that you could start targeting block scoping code in all of the browsers today.
Even if you don't like my particular form, I would still implore you to write code more like this.
Create explicit blocks.
This is totally valid for you to author.
It's your choice.
Do you want to do implicit block scoping or explicit block scoping.
So I would encourage you to at least consider writing code like that.
Here's Traceur, by the way.
They have an online demo that shows things and I want to show you how they're particularly doing their cross compilation.
It depends on your level of choice, whether you think this is more or less hacky.
But they take the code on the left and then they turn it into the code here on the right.
So they're doing what's called hygienic variable renaming.
It's a fancy way of saying-- they're coming up with unique variable names to rename things to.
So they're still using vars, but they're coming up with unique names that's kind of behave like they were block scoped.
It's another one of the hacks and tricks and techniques and if you like that one better, of course, Traceur is a better way to go.
Traceur, up until a couple months ago, was using the try/catch hack.
So there's various different hacks to go about this.
Let's look at some other examples, ones that may not be quite so well known.
These are ones that don't really have a standards track for them at all or aren't really rooted in any particular reality except my own, but I'll just share them as examples.
You don't have to really even understand exactly what's going on, but in this particular example, to express what I'm really saying, what I'm saying with line 2 here is, I'm saying the string foo, the key foo is not in bar.
But you notice how I grammatically said that.
I said not in, but unfortunately, syntactically we have to reverse our brains a little bit and we have to say, not the case that foo is in bar.
We don't have a not in operator.
Except why don't we have a not in operator? Why don't we have a not instance of operator.
In fact, why don't we have and not and operator and a not or operator, because that would make a little bit more sense.
It would create less mental barrier to be expressing things the way my brain is planning them.
Instead of having to force me to go through hoops, I could write code that made more sense.
So why couldn't we just put the negate statement right in front of the in operator? This is a very easy transform.
I've already written a proof of concept of this particular tool to take in infix operators and move them around like that.
So I'd rather write code like this because it makes more sense.
I'm saying not in, and not instance of, and not and because that's what my brain actually means.
I think not only does it allow me to write better code, if we trained ourselves to read that, we would see that that code reads easier.
We don't have to jump through hoops in our brains.
Conditional else clauses on ternaries, the ternary operator that everybody probably knows, the question mark colon operator is sometimes called the conditional operator.
I don't know about your code, but many times I find myself writing in these patterns while I end up having this basically optional thing over here on the right whereas, I'm trying to set something if a condition exists and I can't use the OR operator here because they're actually different between condition value.
But I'm trying to set something, if a condition exists, and otherwise, I just want to leave it undefined.
It's not set yet.
This is a pattern I find very common, especially in my variable parameters.
This is a more robust form of the OR operator, NULL-coalescing operator, if you've heard that terminology.
But I don't like writing colon undefined over and over and over again and essentially, why couldn't I make that optional? I proposed this a while back and they said, they hummed and hawed and said, well, I'm not sure if we can do it.
So I wrote a tool that can do this.
It's just a localized thing that I use myself, but it allows you to skip them if they're optional.
And the tool will simply go in and insert them if you have left them off.
And no, there's no difference in terms of operator precedence, it's exactly the same rules as would apply without it.
This is just a small little taste of the sorts of things that I find frustrating about the language that I'd like to fix.
I'm sure you have your own list and I'm not suggesting that every single problem could possibly be fixed this way, but I think a lot of them could.
A lot of the problems that we've been frustrated with that have been holding us back, we can address them using better tools.
And I hope to inspire more people to write the tools, but I also hope to inspire more people to experiment with what would be better coding for them because that's input that would be valuable for the future of the language.
So these optional clauses would actually be kind of nice, at least for my coding.
Another example, promiscuous this binding.
If you've ever struggled with the this keyword, it does not mean self.
It doesn't mean self.
There is a pre-defined set of rules about it.
It would be nice if somebody would write a book specifically about how the this keyword works.
But it's a very frustrating topic because developers struggle to understand exactly how it works.
There's a related topic to promiscuous this binding, which is the hard binding and that's what you often are faced with when you're passing around functions into event handlers and they're losing their this binding and so forth.
And then there's another much lesser known, but actually, I find this very useful in my code and I call it soft this binding.
There's actually a proposal for a future version of the language that they might try to address this, but I don't like their proposal and it hasn't had any support, so it's probably not going to happen anyway.
But soft this binding simply provides a different default.
Instead of defaulting to the window or global, which is never what we want, it allows you to find your own backup default but still overridable.
You can actually writes soft binding like this and I know that code might look overwhelming.
You don't really need to understand the implementation details.
If you want to, you can find my code online.
But the important part here is I could call soft bind on a function and provide an alternate back up which is still overridable.
It's like the best of hard binding but also it's overridable, so it gives you more flexibility but still it removes that ugly back fall or default that the global object is.
This is yet another thing that I can solve with code but it's ugly and I would like to solve this with syntax.
So I've got a proof of concept that we can do something like use the pound operator like you see here to provide a soft binding hint for a function.
It does exactly the same thing as the previous code, but in my code, I don't have to worry about all the complexities of writing that soft bind and what am I passing in it.
It's a syntactic change, slightly more intrusive, I guess, than some of the others, but it's proof that there are tangible things that we could fix about the language if we chose to experiment with tools.
And finally, conditional catch clauses.
This is something that other languages have had for a long time.
In fact, this is something that Firefox has had for like over a decade, but that hasn't had a standards track and I've been wishing that they would get this on the standards track.
You have this example where you want to catch a clause but only conditionally, only if it's a certain type of error, otherwise, you want to let it just continue to propagate out.
And that's actually a very important concept in certain coding patterns.
The problem is that we have to do this really ugly re-throw thing and in some browsers, that actually can change the way the stack works and what the line numbers are, so that sucks.
This is not a great solution, but it's the way that some people do this.
But even if you were going to do this, I don't want to write this kind of code.
I don't want to have to write that else throw error crap, so it actually is yet another thing that we can fix.
And I'm using, again, exactly the same syntax that was already pioneered with Firefox.
You simply put the if statement right inside of the catch clause.
If error, error if only the error is an instance of type error or error if it's an instance of reference error.
And if it doesn't match, it doesn't get caught, it just keeps propagating.
This is a simple syntactic change and we get the benefits of not having to worry about all those details.
Have it your way.
Have it the way that works best for your brain and let tools do the heavy lifting for you instead of always forcibly having to change the way your brains think.
Sometimes that's unavoidable.
Sometimes you do have to learn things though.
Don't feel like the only option is to just go to an entirely new language.
Sometimes you can just change what you've already got.
So now back to that question that I know many of you have been having about team collaboration.
What on Earth could I possibly do to collaborate with other team members if we had this anarchy that everybody was coming up with their own syntax? How could we possibly do that? And it's a very good and important question.
My answer to that question is under researched, but my answer to that question is what I call defining these things in terms of inversable transforms.
I just made up a term to sort of make myself sound a little bit smarter.
I don't even know if that's accurate.
But essentially, what I'm saying is that all those things that I showed you have two-way bindings.
You can go in either direction fairly straightforwardly.
So a tool could be set up that allows you to have a rule set for what is checked into the git-repo and a rule set for what's shown in your code editor and let the tool handle going in both directions.
When you check out from the code base, it makes the code look the way you want it to look including your spaces or tabs or semicolons or whatever and all these other transforms.
And then when you're done and you're ready to check back in, the tool just does it in the reverse direction.
So what's agreed upon is the rule set that goes into the repo and whatever the developer has is what they get to pick themselves.
And really this is actually trying to solve the problem that CoffeeScript did because CoffeeScript went so far with its changes that essentially this was an all or nothing.
So it's kind of like all CoffeeScript or none.
But I'm hoping that there's a set of changes that we can do to improve the language that we can still collaborate on, that can still have check ins.
And really, to be honest with you, the weak point here is pure coding.
If I have a radically different way that my code looks on my screen and then one of the other developers comes over and sits at my screen, it might look different to him.
But then again I don't do a lot of pure coding, so maybe that's why that isn't such a big deal to me, but I still think there's a way that even if we decided as a team these are the styles and we didn't customize the styles to individuals, but we customize it to the team, you could solve even those problems.
Back to that example.
Just to prove that this is inversable, if you look here, the difference is simply whether or not the colon undefined is there.
It's easy to look for that to parse through some basic grammatical structure and understand if it's there or not and if it's not there, put it there and if it is there, take it out.
That's all it takes.
So most of these rules.
Now, that's not going to let you do everything.
Not every I think and be defined inversably.
So there might be some changes which are destructive or there might be some changes that are a little bit more difficult to build a tool around, but there's a lot of low-hanging fruit that we can fix if we just open up our brains to that.
So this is an open area of research.
I don't have any solid, perfect answers that are production ready yet, but I'm hoping to encourage and inspire people to go further with that.
Now, let me talk about some tools that already exist.
These are concrete tools that already exist that we can use to help us along the way.
I happen to like writing compilers and I'm a geek and I'm weird like that, but many of you probably don't write compilers.
Escodegen. It goes
the other direction.
It takes something that's been parsed and it reconstitutes it as code that can run in the browser.
So you parse code into what's called an AST, it's a fancy tree, you make changes to the tree, and then you run it back through a code generator and you get code back.
And that's the basic cycle that it takes to do coding tools like this.
That's exactly what all your editors do when they're parsing through your code and making type hints and things like that.
They're doing that cycle.
You can do that yourself.
You can build your own tool sets that way.
Escope and eslevels, they analyze the scopes in your code.
And many of you probably don't actually understand how quite nested and how quite complex your scopes are and where all of your closures are.
It would be nice if we could build a set of tools to analyze those things and even solve some of them.
What if the tool could realize that you've got a bunch of nested functions that doesn't actually need to be there, so it could un-nest some of those things which would increase performance and sanity of your code.
We can build tools to do that.
Istanbul is for code coverage.
This is another thing from the testing world, but instanbul will go and tell you, do you have places in your code that you're not even testing yet.
Well, that's a good thing that you need to know and we might even be able to fix that by dropping code that doesn't exist, that isn't being used.
That's a tool that allows you to analyze that tree.
The AST is that tree I'm talking about.
So you can use estraverse along with esprima and escodegen and you can use those tools to understand the code better.
Fantastic there's js.lint and
js.intel out there as well,
but eslint is the most configurable linter that I've seen.
And by the way, just as a side note on linters.
Linters are about style opinion.
They're not about program correctness or validity or anything like that.
They're a set of opinions and this is perfectly configurable to your own set of opinions.
So it shouldn't hurt your own feelings.
And plato.org and
provide tools that actually analyze the cyclomatic complexity.
That's a fancy way of saying how many functions are calling other functions.
Analyze that and perhaps even fix it.
These tools just analyze it, but you could build tools on top of that allowed you to simplify your code without you having to worry about it.
You write some complex code and instead of you having to think about it, you could go through and programmatically figure out how to simplify the code.
So some use cases.
You can auto correct misspellings.
You could auto correct misspellings of variables or auto correct misspellings of keywords or things like that.
You can safely rearrange the scope like I talked about.
You could consolidate your variable declarations automatically.
You could automatically reduce complexity.
You could refactor those terrible, awful Boolean traps that we design into our function calls.
You could optimize the performance of your code and the list goes on and on and on.
And I tried to give you just this quick, little glimpse into some of the things that we might be able to do.
And by the way, there's a couple of really fantastic talks that I think you should definitely pay attention to out there.
So the first one you'll just have to get the slides to get these links, but the first one by Greg here it's a fantastic talk and then there's another one-- I'm sorry the one is by Greg and the other one is by Aria.
So I highly recommend that you check those out.
This is just one example.
But if I do line 6, I get this false false error or a false false value out of it when I just forgot to put the parentheses or if I accidentally put parentheses that I didn't intend.
It's a difficult problem.
It's something that it's difficult to fully get your head around, but I don't think it's that difficult to get a very good working knowledge of.
So I would push back against people that say completely avoid coercion, but this is one of those traps in those problems.
So guess what, tools can fix this too.
Here's just one example.
I love this site.
It's called restrictmode.org,
kind of like strict mode, but restrict mode.
It's a build-time tool that goes through and rewrites your code only for development time checking, but it's a build tool that will rewrite your code with type safety checks around all of the unsafe places in your code.
And then it runs real values through them and it reports back warnings if you're doing things unsafe.
And here is a list of the rules of the tool applies, so the less than operator can only do strings and numbers and if you do anything else than that, it gives you a warning.
So that's it.
I hope that inspires you to think more about tools.
And if you are somebody who likes to tinker and hack, please help build some better tools for the rest of us.
Thank you very much.
I can't believe you sat in the sacred question chair.
I sat in the sacred question chair.
is this real water or is it like stage water? I fucked up.
I don't even know if I could trust it.
Oh, I get my shoes back.
This is awesome.
You ordered a coffee, right? I did order.
Thank you so much.
Thank you very much.
So you were saying we need less standards, better tools, but surely we need better standards as well, right? Like we had promises as a tool for years and years and years, but getting them as a standard, that's what's been massively important though.
So I think it's important-- I'm glad you asked that question.
It's important to clarify.
I don't mean that standards need to go away.
I mean that we can stop obsessing about, well, they didn't put it in the standard, so I can't do it.
That's what I mean.
I mean, that these can develop independently and I mean that what we can do with the tooling can inform what happens with the standards as we've already proven, we just need more of that.
With some of these changes, what was the end game with them? Is to get them in standards or is like the idea is to test the water to see how popular they could be? I absolutely would hope that they would be willing to see if millions of people started using a particular tool and fixing of particular set of problems.
That should be important input and I have every belief that they would probably take that into account.
With particular respect to some of the things I've shown, these are things that I've asked about and have been frustrated about before and been told for whatever reason we're not going to fix.
So rather than taking that as a brick wall that I just have to keep running against, tools can fix that problem.
Are they based on things that you've gone to like the ES list with.
That's what I'm saying.
I've been to the list with some of these ideas in the past and back then I didn't understand any of the spec language, so I got slot down very quickly for not knowing what I was talking about.
Well, that kind of goes back to what we were talking about yesterday and so again, getting involved in specs without having to know how specs work, I suppose.
So that's a similar thing there.
I hope so.
That's that it's really the spirit of what I'm hoping is that this could lower the barrier of entry to some folks who can express in code the sorts of things that they do and would like to do better and use that as an expression for an informal proposal.
Instead of being able to come in and understand all the language around write associativity any and all those other weird things about operators, just express the things that you want to use and the way that works.
And if you have a working tool the proves that it can happen, that's even better.
So with the changes that you had, what happens when it goes wrong, not in the transpiling, but in the usage? How debuggable is it? So that's really all a question about how good the tool is written.
So the best answer that we have right now is source maps.
Source maps are a way to take your original source code and whatever the code that's running in the browser is and map the line numbers and column numbers across all these transformations.
So good tools should be at least either already doing that are on the roadmap to put in source map support and many of them are already doing that.
But source maps are the way that the developer tools and the browser can put you back at the original code that you authored.
So if the tool does that well, then it should be sort of opaque to you that these transformations happen in the background.
And was the tool doing that or that on the roadmap? So for let-er, in particular, there's an open issue and I'm hoping somebody will help me write the source map support for that.
But yeah, I mean, I believe wholeheartedly that that's the story for debugging this code.
So a similar issue to that, it didn't look too bad in most of the examples that you showed, but how much are you going to modify the language until you start losing syntax highlighting? Well, first of all, syntax highlighting is just another tool.
So if you wanted to extend the syntax, you could then extend the syntax highlighting tools.
I wrote a syntax highlighter based on this literalizer code that I was talking about to prove that that's possible to do and if you wanted to and you'd gone significantly off field in terms of your syntax, you simply customize the syntax highlighter to affect the code that you've written.
Would it be possible using the tool that's doing the transpiling to also generate from that the syntax highlighting instructions for popular editors? Absolutely.
I'm not doing that yet, but I think that's a fantastic idea.
Will you put a pull request on that? I'll just code it now, actually.
So what made you write your own parser versus things like esprima and similar things? So in particular, the reason I wrote Literalizer is because let-er needed to parse through non-standard code.
And if you put non-standard code into many of those code parsing tools, it will just choke and say, this is invalid syntax.
And some of them have a loose mode where they just kind of leave it in, but it's kind of dodgy as to how much you can do before it chokes.
So Literalizer sort of is my end around to that for significant syntax changes.
I would try first the loose mode for a change and if that was good enough, I'd use that.
But I wrote my own simple parser.
It's not even really a parser.
It's just regular expression pattern matching and that only works because the complex literals are already taken out of the picture.
But I wouldn't advocate that everybody just go out write their own parser from day one, it's just the step that I had to take to do what I was doing was let-er.
So if this is a parser built to understand new things, are you editing that parser just say for each change you make or is it-- what does it recognize? How does it recognize separate things? So that's a great question.
That's the ongoing area of research that I have, which is right now these are individual, discrete tools that do just one task.
So let-er just does the block scoping and this tool just does that.
And right now that's complex because then you have to have several tools in a tool chain.
This whole FoilScript idea is that eventually one tool could understand all the rules and do all the transformations in one.
That's still just an area of open research.
But sort of proof of concept, I've already proved that individual changes can already happen.
So when you're transpiling from your own thing into ES6 and then down to ES3, can we coin that as the human centipede approach? Absolutely.
I love that.
That's a fantastic way of describing it.
So if I'm using a tool for npm or whatever and I find a bug in it and I think, yeah I've got a little best bet I'm going to try and fix this and help the community.
I go to GitHub and it's written in CoffeeScript.
Not because I dislike CoffeeScript, but I'm going to have to learn it.
Are you talking about massively multiplying this problem? So it's important to understand that I'm not necessarily suggesting that every code repo out there on the planet would have their own willy-nilly versions of code.
I'm more suggesting this is the sort of code that I want to author and then there would be an agreed-upon standard which might be ES6 or it might be your own team version of that standard, but it would be an agreed-upon standard and the tool could go two way between those two standards.
It's possible that you would have two commits.
But you would presumably be accepting full requests into the source file which is the one written in the new style, right? Like that's how CoffeeScript's repo used to do it.
I think that's the most likely scenario, but it doesn't have to be that way.
The only thing after except really is that if you go down this route, it requires the tools because it's not something that you're going to automatically do the hand offering around.
But if you already have a build process and you already have concatenation you're already transpiling, this is just one more step in an already pretty-well defined process.
So earlier Nicholas was talking about the benefits of standardizing on one approach with a team.
Why do you hate him so much? Why do I hate him? I don't hate him at all.
But I do actually find it interesting that there is a constant struggle between this very sort of we got to standardize and make everything opaque, just totally abstracted away versus the other extreme, which is, I want bare bones knuckle.
And a lot of people might look at what I'm talking about as that far extreme.
I still see mind self sort of in the middle because there are definitely abstractions.
You brought up promises.
Promises are an abstraction, but they're a fantastic abstraction and we should build off of them, but not all abstractions are good.
I don't necessarily think that that's an open-ended statement that we should just abstract everything.
But as you say, not just promises, but also what we saw with SaaS.
This is proving that these transpilers are approving out features that actually end up in the main language.
So I noticed that you didn't show any code that made these transformations on how you define them or I didn't spot it.
Why? Is that bad? I didn't show the source code of let-er is what you're saying? Well, not just let-er.
Is let-er the thing that is doing all the transformations you spoke about or is it a tool that you can use to write transformations in.
So FoilScript is this describe your rule sets and that would be in JSON, but it's not finished yet.
And what I'm doing right now is building the actual tools that will do the transformation and then eventually you'll have something that wraps around that with the rule sets.
So I didn't show the rule sets because they don't exist yet, but they will.
So where do you think-- this is kind of more a general ES6 question.
Is there a particular area that you think that they're going in the wrong direction or things that they could change? Yeah.
So first off, again, mad props to the TC39 Committee.
They've done a fantastic job with the vast majority of ES6.
They are completely wrong on the topic of class.
It's a terrible idea.
It's going to make everybody's understanding of the language worse rather than better.
So boneheaded mistake there.
That's one of the rare few areas that I agree with Doug Crawford because he also doesn't like class.
But in other places, there's a lot of great stuff.
--with something like class because I think we do need something to make that prototype of Harrison's a bit easier.
Do you think that's the domain of macros? Is that way you could do that sort of stuff? Actually, I don't think so.
I think object.create
is all we need and that's been built since ES5.
It's the utility that creates an object and links it to another object.
So rather than thinking about designing our code in class abstraction, we use delegation as our design patterns.
It's a very different design pattern.
We use delegation as a design pattern and you just build objects that are linked to other objects.
I call that OLOO, by the way, objects linked to other objects.
And it's a very simple way of doing that, and again, I've written a bunch about that online.
I'll check that stuff out.
--if people are interested.
Kyle Simpson, everyone.