Kent C. Dodds: 0:00 Hello, everyone. My name is Kent C. Dodds, and I'm super excited to be joined by my friend, Artem Zakharchenko. Did I get that right?

Artem Zakharchenko: 0:07 Yeah, it was OK.

Kent: 0:10 Good. Artem and I, don't go back super far. Just this year is when I ran into Artem, and it was through his project MSW, which we're going to be talking about a lot.

0:22 I was just blown away by how powerful and useful this was. Anybody who's gone through the Epic React material, or if you're watching this first, and then you're going to go through it, eventually, you're going to run into MSW on the edges.

0:39 No, we talk specifically about testing with MSW as well. You'll get some exposure to MSW, and this is the person who created it.

0:48 With that, I love to get to know you a little bit. You can tell us anything about yourself, where you work, and what your cat's name is, whatever you want to say.

Artem: 0:59 OK. Hello, everybody. My name is Artem, I'm a software engineer. I'm from a little town in Ukraine called Botella. I've been working as a software engineer for about four or five years.

1:15 I originally graduated like a medical doctor in Ukraine, but then my life took an unexpected turn, and here I am writing software for people.

Kent: 1:27 That blows my mind as a medical doctor background education. I don't know what it's like in Ukraine for being a medical doctor, but I have a friend who literally just a few weeks ago, finally finished all of his schooling and everything after years of going through this. Now, he's finally, an actual, for reals doctor.

1:51 What was that like, and how do you feel leaving that all behind to go totally switch careers?

Artem: 2:02 In Ukraine, I guess it's a little bit different. It takes about five years to study, and then from two to more years to get qualification on your profession.

Kent: 2:13 Isn't that long?

Artem: 2:16 It is not as long as in America. I heard people studying there for decades.

Kent: 2:21 It's wild.

Artem: 2:23 It's been all right. I still think that it's one of the best educations there is, and I'm super thankful. I can't imagine it being more fun and cool, honestly, studying something else.

Kent: 2:35 It is cool.

Artem: 2:35 I didn't switch from it, because I didn't like it or something. I like to do more stuff. It turned out that, I don't know, people needed me for software more than they needed me for doctorship. That's all right.

Kent: 2:50 Yeah, that's interesting. Do you ever plan on going back to being a medical doctor?

Artem: 2:56 I don't like to close myself to possibilities, so may be. At the same time, I'm realistic that everything is practice. Perhaps that's not that likely, but it's nice to know that there is a chance.

Kent: 3:11 That's cool. You got into software, and at some point, you decided that the way that we mark out HTTP requests could be improved. I'll tell you my story of how I find found MSW first, and then I'd love to hear the background story on MSW.

3:33 I found MSW specifically for epic React. I was trying to find a way that I could make an application that didn't require a huge amount of setup on the part of the workshop attendees.

3:47 I've done many workshops. One of the earliest that I did was a testing workshop on the Node side, and I wanted to have a MongoDB and actual endpoints and everything.

4:03 The amount of time that I spent making sure that everybody could get everything installed and it would work was just astronomical. That didn't even work because I had to like say, "OK, go and install Mongo." That was just such a huge ask for a lot of people.

4:20 I know Docker could help me with some of that a little bit. Back then, Docker wasn't quite as well known. I certainly didn't know how to use it and maybe containerizing, that could have helped a little bit.

4:35 I've been burned before, so I wanted something that was simple and didn't require any more additional installation beyond just npm install.

4:47 At the same time, I hacked together my own solution where I just said, window.fetch = my own little thing inside of hijack that. Then it's like, oh, no, now I'm not making network request, so if I try to tell them go look at the Network tab, that's not going to be there. That was a bummer.

5:06 I would do a console.logs and console.group so that you could see those requests at least. It just felt like I'm trying to teach you how to build a realistic application without all the complexities of a real backend, and also, at the same time, I want it to be able to work completely offline.

5:26 If we have to do all these different hacking or workarounds and hacks to make this work, then it just makes the whole experience less realistic.

5:37 That's when I went on the hunt for something and I found Mirage JS, which I'm sure you'd love to talk about that a little bit. They basically were doing exactly what I was doing, just reassigning window.fetch and then doing logs in the console, so you still couldn't see Network tab stuff.

5:54 Then, I finally stumbled upon MSW and it was like, "This is exactly what I need." I still get the Network tab. It's just using Service Worker that just install the thing, and boom, it's there.

6:04 Honestly, the API took a little getting used to, but I got used to it. Now, I love it and it's great. Now, here we are and I'm using MSW for everything. I've completely switched over like, don't mock window.fetch for your test, use MSW instead. That has just been awesome.

6:22 Anyway, that's my story of finding MSW, and hopefully, that gets everybody in the right frame of reference for why MSW is so awesome. I'd love to hear your story of why MSW came to be and what it is.

Artem: 6:40 First of all, I'm thankful and I'm lucky that my dear friend recommended the library to you.

6:47 How it all started. About two, two-and-a-half years ago, I was working. I was just doing React as usual and we were doing some API mocking at work. I guess we use JSON 7, but there were a lot of cases when that didn't work in our particular setup. People were discussing this a lot, and this was in the air.

7:11 I remember that one of my colleagues put down their own solution, which was great, but I was still not fully satisfied with it. I thought it can do much better. What I thought is that, what if I take this idea and I try to make it as perfect as I can in my mind.

7:31 I asked myself, "What is the perfect API marking?" I thought it'd be cool if we didn't have to have any actual servers. We didn't have to change any code to writing stuff. If we're on test then hit this endpoint, otherwise, hit production.

7:46 This was tedious to maintain. I don't know. It would be nice if my request were somehow made. Again, not stabbed and not silenced, if this makes sense, in the app.

8:01 I did some research. At first, surely, it seemed like something impossible, but it was all right. I was reading on the Web, and I found a concept of Service Worker.

8:14 I didn't use it back then at all. I heard it a few times like a buzz word somewhere in forums, but haven't tried it yet. I felt like, hey, this may work because as I understood it back then, it can intercept requests. That's awesome. That's what I want.

8:31 I put down a small demo, didn't show it to anybody. It was a side project, still open-sourced. I guess the first attempt to do that was 100 lines long library, which had a lot of issues, wasn't usable, but it still had this main part working. I was able to make a request, intercept it, and mark it, which was good.

8:58 Then I thought, "Well, hey, my job here is done. Let's move on." I occasionally visited the repo just to do some maintenance, update things. I had some ideas in mind, but as usual, I just postpone it for something more important.

9:15 It was going like this for years until I wake up one morning, and I see that you're tweeting something about me, and I'm like, "What? OK, something happened overnight."

9:26 I checked the history of messages and I see that my friend Ruhai has recommended this project of mine, which I didn't even know it was called MSW back then. Probably it was already.

9:42 I had exactly two thoughts. First of all, this is cool. Second of all, oh my God, I need to redo it right now, because if anybody sees that, it's going to be just shame.

9:57 Then it's been crazy couple of months, just getting feedback from you, getting feedback from some other people trying to use it, creating some issues, and so on.

10:10 I guess during this time, it changed dramatically. It's probably almost a different library right now. It still has the heart and an idea, but a lot of things were rewritten. It was ported to TypeScript.

10:24 There was a lot of stuff handled on the service worker side, because as people may know, that it's not that suitable for API mocking. It has some limitations.

10:37 We did quite a few tasks to overcome those, so people who use it for mocking can just use it and forget about all this stuff under the hood. I guess that was it. It was productive couple of months. I've been excited communicating with you, with others. It's been my biggest open source project, which is awesome.

10:59 I've been open sourcing stuff for a couple of years. It was mainly for myself. I was a little bit burned out by day time and just thought maybe I should just drop this thing. Then this happens, and you never know.

Kent: 11:15 I'm so glad you stuck with it, because it's just been fantastic. One of the things that I loved most about it was, I had been considering writing my own service worker to solve this problem. It was before I had my own window.fetch reassignment thing, and I was like, maybe I just move all that into a service worker.

11:36 Service workers are hard. It's so easy to mess up service workers. One thing that I loved about MSW was the architecture decision that you made to say, the service worker is a pretty relatively simple file by itself, and it's only updated once. Anytime there's a major update then I have to run a little script, which is no big deal.

12:01 It just is sitting there. Then all of my code that I write to interact with MSW is within the context of my application, which was just a brilliant architectural decision. I don't have to worry about the service worker getting out-of-date or anything. I just make a change to my server handlers and whatever, and then it just happens instantly, just like the rest of my application does.

Kent: 12:28 That was a brilliant decision there. Then you added the ability for testing to use those same server handlers for testing. Seriously, that was an enormous positive impact on my productivity, so good cause both of those.

Artem: 12:50 Thank you.

Kent: 12:52 Why don't you talk a little bit about the challenges that you faced with the service worker, because you mentioned that service workers aren't super well suited for API mocking. What are some of the challenges that you faced getting MSW working through service workers?

Artem: 13:14 One of the first challenges there was is how to transfer data. Basically, internally, there's two parts to the library. There is service worker and there is your client-side library that lives in your code as you described, and these different parts need to speak to each other.

13:31 When request happens, worker catches the request, it needs to somehow return the mock response. To do that, it needs to signal to the client that, hey, this happened, do something about it, and then client should resolve this request against your mock definitions, your request handlers, and should somehow tell the worker back that, hey, use this as the actual response.

13:53 I remember it was quite hard at the beginning, mainly as you cannot transfer a lot of data between service worker and client. You can transfer only text.

14:05 I had original idea that I would execute this request handlers in the worker but they will be defined on the client, so I'd somehow pass the reference, call it there, and it's going to work. Obviously, it didn't and never stringify functions. Thank me later.

14:23 Then I was just reading some stuff on how workers operate, what are the APIs there? Apart from activating, installing, updating, and intercepting requests, there's also a few things you can do.

14:38 I didn't know that you can do communication between worker and client, something similar to inter-process communication, where you can send signals and you can receive signals, which was good.

14:52 I started digging into that. I found out that you can also send some data, you can send some strings. That probably was the moment when I realized that, hey, this is very cool. This may work.

15:04 I was excited about that. It felt like cheating, I didn't know why. Maybe because I didn't know about this possibility, and then suddenly, I was able to do exactly what I needed in some minimal way, of course. That's probably what defined the architecture.

15:21 I wasn't that experienced with service workers to draw some conclusions like, hey, this should execute here and this should be signal there. I was trying to break service workers so it suits my need, and I guess the spec was very good. It guided me towards right decisions.

15:40 In the end, it was just a logic that would stringify some information about the request and send it to the client, and the client will do all the heavy lifting, and again, just send the stringify response back because that's all strings, and you didn't have to pass any hacky JavaScript around. That was quite interesting in the beginning.

16:04 Then it was a lot of things during our cooperation. I guess there was some issues that you need to update the worker on reload if you did some changes, and for some time, it was annoying.

16:25 This doesn't come enabled in browsers. You need to check it in the browser basically, it's not a code feature. Asking to check it felt like a huge prerequisite to have, plus it behaves a little different in different browsers. I'm not even sure if some browsers have this option.

Kent: 16:43 Yeah.

Artem: 16:43 I don't remember how exactly I solve that. I guess it came to the concept of self-updating worker or something. The library could listen to update event and then if update happened, it could just, I guess re-install the worker or update it on the fly, so we didn't have to have it enabled in the browser or anywhere.

17:09 One of the recent challenges we faced just this week is that there's been updating from, and perhaps, some other browsers that kills service worker, if it stays inactive for some period of time.

17:25 After 20-30 minutes, if you're just switched from your app doing some other stuff and you come back to the application, everything can just stop working. The request would fail. It was a bummer.

Kent: 17:38 It seems a pretty significant change. I'm sure it's impacting a lot of people now.

Artem: 17:44 Yeah. People raise some issues, and thankfully, one of our contributors went in and did some research and resolved it. I believe that we published a fix today, so it should be very good now.

Kent: 17:58 Great. I'll make sure to go and tame everything. Thank you for managing that for me. If I'd done that myself, I would have to do all that research. This is the beauty of open source. Thank you, and that contributor.

18:13 I recall also, having service workers hang around. I would have one workshop that used the service worker, then another one that didn't, so I'd close this one and then I'd start this other one up, and I'd still have the old workshop, and you solved that, too. How did you do that?

Artem: 18:33 The root cause of the issue was that when you register service worker, it controls what is called clients. You can imagine it like browser tabs, because I believe that is a client, and it controls them by their URL. If you have different projects on the same URLs, you may want unexpected behavior, like mocking kicking in when you don't want it.

18:57 It was solved by explicitly removing the worker when you close the tabs.

19:06 When your last client closes, the worker understands that and it can self-destruct itself. Then you didn't have this issue, because if you go to another project on the same address and it doesn't have MSW, there is nothing to register the worker, which is good.

19:30 Otherwise, they persist between sessions and so on, and you don't want that, in most cases. You want that in case of caching which worker is designed to do, but in API, that maybe not the best thing to have.

Kent: 19:44 Precisely. Honestly, there's part of this. We're definitely using this API for what it's not intended for, and so there's some workarounds and things. At this point, it is solid and I have total confidence in MSW. It's been remarkable what you've been able to accomplish with that.

20:07 Another thing that I wanted to ask you about was anytime that I've gone into MSW's codebase to research, whatever, I found your test suite is really good. Testing service worker is something I don't have experience with. I've never done that.

20:28 I'm surprised at the thoroughness of your test suite and I'm curious how you accomplish that. I see stuff with Puppeteer and things, but how do you get that?

20:38 If I were to build this myself without diving into deep research, I'd probably just try to break up the library into small chunks that I could just unit test, then pray that they integrate OK. I'm curious how you get such good confidence from your test of service workers?

Artem: 21:00 I still have it split into small chunks and there are a bunch of unit tests, of course. Then I've been reading a lot of materials about testing as well as some articles you put up, which is awesome. I just decided that the only true way to test that it works is to be as close to what actual developer does.

21:20 What they do, they open the browser, they load some JavaScript. JavaScript registers the worker, then they do some idle interactions, they do requests, and so on. I guess it was just natural to try to spawn some browser during tests and try to register the worker, and so on. It surprisingly was not that bad.

21:41 I chose Puppeteer. I was considering also Cypress for that. At that time, I chose Puppeteer because it allowed me to control network to the way I wanted. You can set up your own interception in Puppeteer.

21:57 What was needed for the test is have full control in listening to actual request that happen, action response that you received, you can get all the headers and stuff, and you can do it on the PROM level, if it makes sense.

22:12 It's not your app doing this, it's Puppeteer somehow communicating with Chrome, and Chrome lets you know, hey, this has happened. It was quite nice.

22:23 Then I was just sitting one evening and thinking, how can I make it better? I love the idea when my tests utils also usage example. I've been trying to practice this for multiple projects, and here, it applied pretty good as well.

22:39 What I did is that I created small tests utils, which had probably nothing related to test, but they were usage example. It's a regular JavaScript module where you input stuff, you do some actions and stuff like that.

22:54 Then I created some setup code, some environment that can pull this code in, create a dev server, then make Puppeteer open this dev server, and then serve MSW like the queue run build you have, and it would just work. It's been really useful.

23:12 We had some issues with the vApp active server. We configured it wrong, and then it took some time to figure it out right so there is no memory leaks and so on.

Kent: 23:25 It takes everybody time to get anything to work with Webpack. This is why I'm so glad we have these frameworks now that I can capsulate Webpack. Anytime you have to dive in, that's a trick.

Artem: 23:39 Yeah.

Kent: 23:42 That's awesome. Your Puppeteer tests are they running in your own little Node script that you've created, or is that part of Jest? I can't remember whether you have this all running within Jest or not.

Artem: 23:56 Let me remember myself. How it works, it runs with Jest and then I have a setup command that accepts the path to this use case. What it does, it spawns Puppeteer, it spawns dev server, it orchestrates their communication if needed, allows to close them all off to tested on, and you call it in before all or before each in each test suite.

24:23 What it does, it just runs the test, it sees that it needs to spawn the browser, it spawns the browser, it communicates with the dev server so it basically opens it like an app in this browser, and then it does whatever interactions you need to do. Usually, I just perform request from within Puppeteer context, and then I see what happen to that request.

Kent: 24:44 Now, that's cool. Awesome. I'd like to take a little bit of a step back and just ask what your thoughts are around API mocking for both our development's experience as well as our testing. Why do you think it's so useful to do this?

25:07 You mentioned earlier that you don't want to have code in your application that says, hit this in development and this in production, wherever. That makes sense to me. Why is MSW a good choice for mocking? Maybe even to take it even further, why should we even do mocking? Why can't we just hit the production server or maybe a staging server? What's the benefit of mocking at all?

Artem: 25:35 There is a lot of levels where you can apply mocks, like from mocking specific modules to mocking API requests. When it comes to mocking, I tend to think that if you can avoid it, avoid it. If you can construct a function, your component, your part of the app the way that would maybe allow you to test it without doing mocks, it would be great.

25:59 For some things, you clearly can't. When you do a HTTP request in your app, it needs to happen. That's the part of functionality so if you omit it, you will be losing. You will be testing a different code, effectively. It's not going to be your app code, it will be a little bit altered code, which gives you much less confidence, I suppose.

26:22 What shines for me and where I see MSW useful is that when you apply it in tests, you have this invisible layer on top. This layer is part of your test, not your app, because you can set up the way that your app will not even know anything about MSW at all.

26:46 It will be great because if you have two different parts, your app and mocks, and you need to try hard to make a code leak from one part to another, you would end up with more strict tests. Your app will not do something it shouldn't do like rewrite fetch or XMLHttpRequests globally, and your mocks will also try to behave like an actual server. I don't know. Perhaps there is some limitation here as well.

27:16 I guess that's the main benefit to be there. If you create these deviations as I said, for the sake of testing or for the sake of development, it may work. At some point, it may get either expensive to maintain, or perhaps it would get some false positive result sometimes. It depends.

27:39 I'm not a fan of stabbing things, although MSW does do if you use stabbing things when it runs in Node, to be clear. Still, I feel like if we can do it somehow without touching actual code or even the standard library, it would be amazing. That's what we're trying to do.

Kent: 27:59 That is a good idea to try and avoid stabbing things, so that the code at least is running as close to the way that it runs as possible. I agree with you that there are just some situations where you can't avoid mocking. You wouldn't want to charge your credit card when you check out during tests or something, or even during development.

28:27 Having MSW as a solution can help us sidestep some of the issues that come with mocking because the way that it's implemented. At least for me, it speeds up the development time, my productivity, which is just so nice. The ability to be able to work completely offline.

28:52 I was going to use PayPal as an example, but pretty much everywhere I've worked. When you're doing development, you didn't run the entire app on your local machine, you would hit a staging server or something, and if that staging server ever went down, you can't work. I would have appreciated having MSW.

29:16 Then to take it a step further, I know a lot of people like JSON server or something like that for a mock server. What would you say are some of the benefits of using MSW over hosting a mock server locally?

Artem: 29:34 Initially, if you host another server, it comes with the cost of having to run the server, potentially maintain it sometimes. It's basically a real server, which you just downgrade to be an API mocking server, and it may be suitable. It definitely has some usage scenarios. I don't know. I don't see anything bad in that, particularly.

29:57 If you want to do it a regular basis, especially if you want to integrate it with tests, it may be costly to spawn the server each time, clean it up. It depends on you. I would personally try to keep my testing setup as easy as possible. Spawning another server, maybe not. That's a good decision for me.

30:20 I guess the biggest advantage is that if you have a dedicated API server, something in your code needs to hit it at some point of time. If you just keep speaking with some stage server or production, it's not going to happen.

30:35 You effectively go and you change your code, which I'm trying to set a taboo for mocking. You shouldn't change your code for the sake of mocking.

30:44 Even for the sake of testing, of course, it comes with extremities. That's a nice thing to have like a guidance. I like when tools encourage you to do that.

Kent: 30:55 Yeah, that makes total sense. Love it. This has been a great conversation. I had a good time. Is there anything that we didn't talk about that you wanted to make sure we cover?

Artem: 31:08 It's all right. I just would like to give a huge round of thanks to everybody who contributed, including you, first of all, and a lot of other people who raised issues, suggested features, tried it, most importantly. I'm happy to the state we got the library to. That's just awesome.

Kent: 31:28 That's good. Hopefully, at this point, there's not a whole lot more that needs to be done that will take you a whole lot of your personal time, because I know you're not necessarily using this at work. This is your personal time.

31:44 We appreciate your personal time, and hopefully, we can get more contributors to help take care of other things like when Chrome decides to delete the service worker after 20 minutes of non-use or whatever.

31:58 Thank you so much for all of that effort and work that you do. It's seriously made me a more productive engineer, being able to use those handlers during development and tests if I have a storybook. I just don't need to worry about any of this stuff. I just have it all set up in one place and it works brilliantly. Thank you for that.

Artem: 32:20 Yeah, I'm really happy.

Kent: 32:22 OK. Cool. Thanks, everyone. We'll see you all later.

Artem: 32:24 Yep. Take care.