Recorded 2021-10-21. Published 2021-11-12.
Chris Smith is interviewed by Joachim Breitner and Andres Löh. Chris is the author of the CodeWorld teaching tool and discusses why too much curry in the language can make error messages hard to digest and why a self respecting testing library certainly should be used to test itself.
Joachim Breitner: Welcome to the Haskell Interlude episode with Chis Smith author of the CodeWorld teaching tool. Stay tuned to hear why too much curry in the language can make error messages hard to digest and why a self respecting testing library certainly should be used to test itself.
Andres Löh: Hello!
JB: and our guest Chris Smith calling in from Atlanta.
Chris Smith: Hello!
JB: Chris Smith has built CodeWorld which is a place for people to learn about computational thinking and math and also Haskell and I want to note to so get started with how did you get to know Haskell and what made you be involved so much with Haskell.
CS: Oh yeah, I actually found Haskell somewhat indirectly and somewhat early in my programming career. I was trying to branch out and learn new things and I ran across video recordings of Ableson and Sussman’s SICP lectures. So I started watching these with colleagues from work at the time and they started talking about this language Miranda which was really fascinating to the things that they said about Miranda sounded really attractive and so I went looking for this programming language. Well this was in the 2000 so I did not find it but I did find that its sort of successor in some sense was Haskell. So at that point I started learning Haskell just as a way to broaden my own understanding of you know of the world of the programming language world in particular. Um, so that’s what brought me to the language.
JB: And how did you get from there to building CodeWorld? What is the the origin of CodeWorld or maybe what is CodeWorld first us so that people who don’t know it in the audience will recognize this.
CS: Yeah, yeah, sure so what CodeWorld is? It’s a web-based platform built on top of GHCJS so cross compiling Haskell to JavaScript. Which lets anyone sort of go to this webpage write some Haskell code without installing anything in advance click run it sort of builds into JavaScript code and runs in their web browser and it comes with a graphics library so that it’s really easy to do really simple graphics programming and some other things like that. Ah basically the idea is to be able to get some like high impact graphic you know, graphical exciting code with a minimum of upfront investment. Not just that though. It’s also designed to really emphasise the kind of mathematical thinking and algebraic thinking background of the Haskell programming language and the reason for that actually is a great transition to where CodeWorld came from. The story behind the CodeWorld environment comes down to my neighbor who lived next door to me when I lived in Colorado. She was a teacher at a middle school and at one point she decided that she wanted to leave the school that she taught at and start her own school in her own living room next door to my house. So I talked to her about what she was doing and she was really passionate about teaching history and social sciences and reading and literature but she sort of came to me and said well what can I do with mathematics that makes it not just working through math worksheets. But doing something that really teaches a fundamental understanding of mathematics and so my first year that she was doing this I went over there a couple of days a week and I taught a class on recreational Mathematics. So we did a lot of sort of traditional proof-based mathematics in terms of exploring the Towers of Hanoi. Exploring you know the art gallery problem. You know things graph theory and combinators problems and things like that.
JB: What is the art gallery problem?
CS: Oh the art gallery problem is the question if you give me an arbitrary polygon with n vertices how many points what’s the the most points that I may need to pick such that there is a line in the interior of the polygon from each point to any other point in the interior of the polygon. So the idea is like suppose you made an art gallery with some polygon shape. How many places do you need to post guards so that they can see the entire interior of this art gallery.
JB: Okay, so what is the age group that you are teaching to.
CS: Yeah, so the school was interesting. It started it only had 6 students to start with and the first year it was sixth grade which is 11 to 12 years old the next year it was the same students in seventh grade and the next year it was the same students in eighth grade and then it went back to sixth grade again the year after that so it sort of cycled through this 3 year period so that any set of students could be at the school for 3 years,
JB: But there was still just math that you were doing teaching there.
CS: Yeah, so the first year it was it mathematics and then towards the end of that year I was just thinking through the question of what’s a way that I can take mathematics. It was a lot of fun to run these recreational mathematics problems but it was also a lot of very top-down learning. It was very much I would show up 1 day and tell them what we were going to talk about this day and teach them and I wasn’t really happy with this and so I thought what’s a way that you can do mathematics but you can do it sort of in a self-led creative manner. And so this is where CodeWorld came in and the idea that I try to make underlying the environment is that it’s computer programming using algebra and if you think about it at the sort of very basic level that’s sort of what Haskell is; You know writing essentially just algebraic equalities they happen to be defining something on the left hand side of an equal sign but you can think of them as actually making statements of equality that that result in those definitions and then you know by doing so you can define what you want. So the idea was to try to keep computer programming as declarative as mathematical as possible and to provide an environment where essentially you’re just writing out algebra. But it’s computer programming. Haskell turns out to be a great fit for this style of programming. And not just Haskell but there are some specific techniques like the gloss style graphics approach where you have you know a state transition function and a function from state to picture and you just iterate these functions in order to produce an application so within Haskell this was popularized at a library called gloss outside of Haskell it was popularized in racket with it’s called the universe model I’ve heard it called many things in many different functional programming communities. I tend to call it referred to it as functional mvc or functional model view controller as ah, a sort of generic name for it. But yeah, that’s basically the idea is can we take a programming language and it turns out Haskell was ah a nice one because Haskell is almost there already and turn it into something which is just writing down algebra as a programming language.
AL: So was it immediately clear to you when these ideas started forming in your head that you would have to design something like CodeWorld or did you first think you could somehow just use Haskell directly.
CS: Yeah, so the first year I did just use Haskell directly and the resulting software that I wrote looked nothing like what I ended up with. It actually compiled and ran applications on a server and then streamed frames from the server to the client and streamed UI events from the client to the server it was built on top of this extension called Safe Haskell which allows you to sort of run code on the server but control what it does so there are a lot of like of little bits like this ah and you know around the so and initially it was a very different architecture but and it was very much straight Haskell as well. Um, it was a sort of second iteration of this. So after that experience I taught with this for several years in Colorado and then I moved from Colorado out to California to work for Google and while I was working for Google after a year or so to acclimatise myself to a completely different culture I went back and thought through if I were doing this again what would I do and so I originally built a second version in something called faith which is one of Chris Done’s projects ran into some limitations there and was encouraged to try GHCJS and I was able to build something on top of GHCJS and then at the same time that’s when I started thinking more about okay I have Haskell okay. But there are a lot of things in Haskell that are not part of the learning objectives of learning mathematics and so can I come up with a simpler programming language and because I did not want to write a new compiler. My constraint from the beginning was it will be Haskell but it will be Haskell with simplifications simplifications imposed on top of it.
AL: What kind of simplifications.
CS: Yeah, yeah, what kind of simplifications? Number one is that I rewrote the entire standard library using the Rebindable syntax extension and a bunch of new types so that you’re not dealing with a lot of the legacy craft around the string data type for example. Also so that there’s just sort of Javascript style. There’s one number data type which happens to be a double but represents exact Ints at small enough ranges. Because that it certainly avoids a lot of the complexity of overloading and type classes things like that. I committed some horrible Haskell atrocities that actually helped me out by writing a JavaScript function that would implement arbitrary equality comparison of values with no type class constraints by walking the Javascript heap representation used by GHCJS in order to again, get type classes out of this. Type classes, you know are very powerful and a great technique but they are the start of where Haskell starts to get complicated. And so I wanted to stop short of that. Another change that I made was to uncurry the entire standard library and that was actually based on my experience teaching from my original implementation. I had learned that a lot of the most confusing and complicated error message situations came from forgetting a parameter and then having the result misinterpreted as some higher order function which was partially applied so undoing that currying in the standard library and writing a completely uncurried standard library gives the language a completely different feel but it’s all still Haskell. You can do this and it’s valid Haskell it just is Haskell with constraints.
AL: So you’re basically reusing the entire front end of GHC still. It’s like the original parser
CS: Yes, the current state of CodeWorld is that it runs a sort of a preprocessing step where it parses the code in GHC’s parser and then uses a bunch of like scrap your boilerplate rules and things like that to match certain patterns and reject them in advance before it will actually even try to compile the code. But if you pass all of those rules eventually it takes your code and just passes it along to GHC.
JB: And these rules are only predicates to check something. They don’t change the code. You’re not preprocessing in that sense.
CS: For the most part yes. The one place where I ended up having to change the code is I wanted to do. Oh this is maybe a little off in the weeds. I wanted to do this feature called Import by hash. One of the the features that has been really interesting in CodeWorld is whenever you run a code run a program it just hashes your source code for the program and builds you a url with this hash in it, which you can then refer to to share the program with other people and so on. I wanted you to be able to also take that hash and say import the code with this hash but then that did require making some modifications to the code for some just complicated reasons where you know if you had declared a module name or the module name had to be the hash of the source code. But of course the hash of the source code depends on the module name. So it uses the hash of the original source code but then modifies the source code to change the module name and so on. But that’s a corner case in which the code is modified but outside of that import by hash feature that is absolutely true that whatever code you write is the code that is given to GHC.
JB: So you said you uncurried because it’s confusing when you leave all the parentheses. But I expect that well okay, now you still leave all the arguments you still get an error message that is completely unhelpful and tells you something about higher order functions.
CS: Well well it doesn’t tell you about high order functions. It actually comes in and tells you hey I expected the type number number text and I found number number.
JB: Oh but if you completely forget the argument list then you still get the confusing error message.
CS: Ah, that is true. Yes. So if you write the function with no arguments but remember that if we’re used to currying everything then we’re actually back to the sort of mathematical way of writing functions. So actually applying a function with no arguments you’d expect to be like f empty parentheses and then you’re back to getting this error message that says you know hey I found empty parentheses unit and I expected these these arguments. So only if you leave off the parentheses entirely, do you end up with the higher order function.
JB: Okey
AL: And I can certainly confirm from my own Haskell teaching experience that getting students to just drop the parentheses around the arguments of functions is one of the hardest things that you have to initially do so I’m not surprised that it is relatively easy to get students to write them if you actually want them to do so.
CS: Yeah, and that was actually another part of the reason for making that uncurrying decision is that I actually wanted students to feel like they were working with math in the notation that they had already learned in school and so that does involve parentheses around function arguments. Even if it’s just one function argument so actually one of these rules that I enforce before I will send your program to GHCJS is to ensure that anytime you have a function application that there are parentheses around the argument even if it’s only a single argument.
JB: So given all of these list of things in which CodeWorld when you open it by default differs from Haskell I’m too big of a fan of both CodeWorld and Haskell to not point out that there is a mode where you can still get the raw Haskell experience and just as a like a shameless plug here and there I’ve written a tutorial about teaching basic Haskell by implementing a game called Sokoban in CodeWorld in the normal Haskell mode so you can use it to learn Haskell and you get all the horrible error messages from GHC and you get to deal with type class and it’s all there so.
CS: So to be clear if you go to https://code.world/, you will get everything that we’ve been talking about in this podcast where you’ve got this modified simplified version of the Haskell programming language. But if you go to https://code.world/haskell then you will get the the exact same tools, the sort of web-based environment in which you can write your code and run your code and see it in the browser but everything is back to the standard Haskell prelude and a version of the graphics library that follows Haskell conventions and things like that. So yeah it’s really important to me that you can do both that I’ve got this sort of you know that there’s the version that’s intended for teaching younger students and then there’s the version which is actual Haskell and I’m certainly not opposed to Haskell I think this is something that I’ve run into a few times where people have said you think Haskell is too complicated and needs to be simplified. It’s like only for my 12 year old students who are there to learn mathematics and not to learn Haskell. I’m a big fan of the Haskell programming language. It’s just that there’s a question of audience and goals.
AL: To turn this around briefly is the simplified language also available without the web-based environment?
CS: No, it’s not. It would be interesting to look into that I think one immediate problem you would run into is that the way that we implement equality comparison depends very much on the fact that you’re using GHCJS as your compiler.
JB: But that could be solve. We have this GHC heap view library in GHC which inspects the heap and you can just distinguish the data constructors and as long as you don’t care too much about what equality does on functions. Um…
CS: Yeah, that would be great and in fact I know that there are bugs in the GHCJS equality comparison. So if you have an idea for how to fix that by all means I would encourage you to talk to me about it because I think like I said the equality comparison is actually subtly broken in some cases right now that is an open bug with CodeWorld. So I would love to get a sort of more correct implementation that would be guaranteed to work and.
JB: Well, the other rotation is doing the same horrible hack just doing it on the running heap of a like a natively compelled Haskell program so it wouldn’t help you in the GHCJS environment and you would probably get a different set of obscure bugs, which may or may not be better, but it could be good enough for people who really want to run this locally so that’s I think that might be an interesting project.
CS: Yeah, I think in some sense the goal comes into mind here where if your goal is to sort of teach a computational thinking sort of class which is the goal behind the educational variant of CodeWorld then in some sense you kind of don’t care about getting close to the Haskell programming language because teaching Haskell is not your goal in the first place. If your goal is to teach Haskell then ultimately it feels like you’re using the webish environment as a crutch to avoid getting stuck on installation in the first few weeks right? But if you’re teaching Haskell you eventually do want your students to have installed their local tools and have the normal tools that’s available for the Haskell programming language. So there’s a bit of correlation between the goals.
JB: Yeah, and you can get as far as I know the graphics part of CodeWorld as a native library.
CS: Yes, so there is a library that implements the graphics API for CodeWorld called codeworld-api and you can definitely write your programs using that API and then build them locally using native Haskell using that API. But that’s only for programs written at the CodeWorld/Haskell variant of CodeWorld. If you have the educational variant then there is right now no way to install those build tools locally.
AL: Okay I would like to go back to the to the teaching aspect actually. So you said the first few years you just used Haskell and then you set out to implement code world and did you then get the chance to actually try this out with students and did you actually experience that they react to this new environment in a better or different way?
CS: Oh yes, absolutely so I’ve spent more effort teaching with CodeWorld than I have writing you know, building CodeWorld. Not lately because covid19 makes it difficult to volunteer in schools. But prior to that I was always volunteering in at least one school most of the time to teach using the CodeWorld environment and a lot of my decisions were guided by my experiences doing that. So I will say that when teaching with just plain Haskell it was very much one of these things where we were back to everything was very top down instructor led. Someone would have an idea and I would sit down and pair-program with them how to get that idea from their, you know drawing board, to having a working program but I definitely did not achieve the goal that I had that students could sit down and build something you know, build more of their own ideas after they left the classroom and so that was really my main motivation for changing things around was that that’s. That was the goal that I wanted. I wanted not just to have students who could sort of pair-program this idea with me and then be happy that they had created something but you know in collaboration with an expert. I wanted to get them to the point where then in the summer after they had worked with me, they could sit down and they could make something different and this is an experience that I’ve had I’ve actually had students now send me animations or things like that that they have made in CodeWorld, entirely on their own after my classes have finished and that’s that’s a great thing. It makes me very happy.
AL: And what do you think is the decisive or the most decisive difference that enables the success? I mean is it the environment? Is it the error messages? Is it like um.
CS: Oh it’s a combination of a lot of this. I think the error messages are maybe the smallest piece. The choice of a very simple API I think is a really important step in that direction. And I know Joachim has worked with me, we actually wrote an ICFP paper together on building an API for multiplayer gaming that sort of follows this very simplistic and very declarative model and it was interesting how many constraints that put on us in terms of you know, it was like the the there was just barely this narrow path for the API to actually be implementable and still look like we wanted it to look and that that kind of simplicity actually turned out to be really important and then I think just the little details of things like you know making currying match familiar syntax things like that play a big step as well. Though, I think the API is the biggest part.
JB: Do people actually use this multiplayer online gaming thing.
CS: Oh people did. It’s not something that I’ve pushed or taught with lately but it has been something that I’ve done a little bit with. I will say that most of my teaching experience ends up stopping on the drawing and animation side of things because there is something about building stateful games with state transition functions and things like that which I think it could be really successful if you had a full school year to teach it. But that’s a lot of time to wrestle out of the hands of a school district to be able to say let me do you know daily class for an entire school year just on this one activity. It’s a difficult case to make and so what I ended up getting instead is offers to come in once or twice a week for a semester or something like that. And then there’s you know at the ages that I’m looking at you just need to keep things simpler so in some senses I rarely have gotten to the point of anyone using this stateful game API in the first place and therefore I haven’t gotten people to use the multiplayer game variant of that. But I think the multiplayer game variant is not that big a step over the single player game variant.
JB: Yeah yeah I have a vague recollection of sitting in a pub in Oxford after ICPF with somebody else I forgot who it was and we were implementing a collaborative text editor based on this where you could both type and it would like have two cursors and it would just resolve conflicts because that’s part of the the idea behind this lock step simulation mechanism it kind of worked so is it definitely fun to play around once you are past this step where a state transition function is no longer scary but that isn’t the target audience. Maybe people going to ICFP.
CS: Yes, right? It was definitely fun to build though. So I don’t regret it.
AL: Basically one lesson to take away also in general is if you want your Haskell libraries to be successful is invest a lot of effort in a good API I guess.
CS: Yeah I think that’s actually a great lesson to learn.
AL: And you use that so abstractly now but can you summarize what sort of makes an API easy to understand and easy to use and in your opinion or what are your experiences.
CS: Oh yeah I think what sort of guided my CodeWorld API implementation was just this notion that everything had to be as declarative as possible and then that actually has surprising you know implications once you start sort of drawing that through the whole process observing how people respond to the API. You know I sometimes looked at it as I was you know doing this in more of an experimental way that I was looking at how the students that I was teaching interacted with the API and then I was making changes to suddenly discourage the things that I didn’t want them to do and to encourage the things that I did want them to do. I think one example of this that’s come up in a conversation recently is you know pictures in this API form a monoid where you can take two pictures and combine them which just overlays one on top of the other and initially I think if you think about this from a very imperative standpoint you think of the painter’s algorithm where you paint the stuff in the back first and then you paint the stuff in the front on top of it and so you expect like picture a combine picture b to b picture a in the back and then picture b is drawn on top of that. But it’s actually the other way around because I don’t want someone to look at this and think oh this means draw picture a and then draw picture b. I want someone to think this means it’s a description of the picture that comes in from a combination of this picture and that picture and so if I put picture a in front and picture b in back then that is a perfectly good semigroup as well it’s a perfectly good operation but it’s one that does not encourage you to think of this operation as “and then” right you know draw this and then draw that. Instead it encourages you to think of it as like this is a complete picture as a declarative value, not as a set of instructions for drawing a picture but as a declarative value and that’s a picture as a declarative value and then put this one in front of that one.
JB: So you spend a lot of time thinking about how the language should look like for this particular target audience and you said that, well this is not the ideal Haskell but truly some of these things you could probably imagine being also good for Haskell as a language. So imagine we would replace the GHC Proposals Committee and the libraries committee with you being dictator. Which changes would you make to Haskell.
CS: Oh that’s a scary thought indeed… Which changes would I make to Haskell? I think that ouh, my gripes with Haskell tend to be mostly in the lines mmm….
JB: But let’s start with something easy. You would probably I would assume you would say well text is definitely a better type than list of characters for textual data. That’s an easy one right?
CS: Yeah, yeah I would say that some abstract text type would be an improvement over a list of characters that doesn’t feel like a controversial point of view really? I think it would be difficult to find people who disagree with that.
JB: I think, yeah I expect that the main disagreement is about it’s just too much change is too disruptive. But if you ignore that as well and you really get to say: okay, that hassle should be on and people for some reason don’t worry about rewriting their code to adjust. So okay, we we get a better type for text. So what else?
CS: Yes, yeah, look. That’s a small one. I think huh… I tend to not have a lot of opinions about this honestly, specifically because I have spent so much time thinking about something which is not the core use case for Haskell and so I do have opinions about that because that’s something that I’ve observed that I’ve experienced when it comes to observing other Haskell programmers my experience is actually surprisingly limited. So, it’s only the last month and a half where I have worked with other Haskell programmers professionally in any way. So in some sense I’m still not sure I have my head around the professional Haskell picture.
JB: How does it feel like suddenly working on Haskell again in a professional capacity.
CS: You know it’s really exciting I think it. It’s really more for me about the project than it is about the programming language. But I do think that there’s a correlation between the choice of programming language and the way that people think about the questions that they’re approaching. So I’ve been working at Groq which is this hardware company and working on their sort of Assembler/Compiler Infrastructure, we call it the assembler but it’s more than an assembler. And the idea is to sort of be able to take I mean the the idea of the whole piece of the company that I’m working in is to be able to take ah you know machine learning models and turn them into instructions that run on this this piece of hardware and, you know It’s been an interesting adaptation because I’ve never worked with anything at this sort of low-level hardware level before so a lot of things that I have really only ever seen in a computer science class. You know one computer architecture class at a University and I’m now actually seeing on a day-to-day basis. Has been an interesting adjustment. It’s also been great in the sense that, and this may not be Haskell specific, but I think it’s like it is in some ways Haskell specific as well and that is I think like working for a large company with a lot of internal infrastructure one gets to depend on a lot of internal infrastructure and just assume that the things that you’re doing are being done with internal tools that have no relation to the world outside of your company. I think like one of the things that’s been really nice about working in a Haskell company is that there’s more interaction with the broader Haskell community, so I’ve actually been able to do a lot more upstream contributions to Haskell libraries, work on Haskell Language server and how it handles some things, things like that in addition to the internal software that I’ve been writing.
AL: So now I have multiple questions at once actually because you’ve given so many keywords here. So I’m wondering in which order so one thing that I wanted to ask about which still relates back to CodeWorld actually but you mentioned Haskell language server right now and to some extend it feels like there are obvious connections between. I mean CodeWorld is also an integrated environment that does sort of on the fly feedback and so on and so forth. So is there anything that Haskell language server can learn from CodeWorld or is there anything that due to the presence of Haskell language server you could replace on CodeWorld and make much simpler? I mean is there any kind of synergy between these?
CS: Yeah in some sense the two have picked very different directions to go in solving the same set of problems. CodeWorld because I’m ultimately I’m trying to keep as much of it as possible running in a web browser, I’ve ended up taking the direction of reimplementing bits of Haskell infrastructure in Javascript. Ah, and that’s been the sort of approach to solving a lot of the environment feature problems. So like in the CodeWorld editor, for example, as you’re typing, the editor keeps a running dictionary of what modules have been imported and what symbols are available from those modules so that it can implement autocomplete things like that. In Haskell language server this of course lives on the server which is typically local host for Haskell language server because you’re normally working on a locally installed environment and so yeah, all of the stuff live on the server and it involves actually keeping open GHC sessions, things like that, which are in my mind sort of out of the question for CodeWorld. Because CodeWorld does not want to maintain the expense of having an open GHC session for every concurrent user that would be I think like one of the problems that I run into with CodeWorld is that first day of class where I tell everyone okay type in this first program and now hit run and then that instant you know I have a class full of sometimes you know 60 students and every one of them is hitting the server asking it to compile this program at exactly the same time and you know I have often scaled up resources for the server or or added replicas of the CodeWorld server before my first day of class for this reason. So if every time someone showed up in this environment and wanted to start writing some code I want I had to maintain sort of a new GHC session for them I think that would just get too costly. On the other hand with Haskell language server you assume that the language server is running on local host that the local host probably has nothing better to do and therefore you know it’s essentially free to just maintain a GHC session and you know get this information directly from GHC and allows them to do a lot more powerful things but I think things that ah, that would not scale cheaply enough for me to do them in CodeWorld. So I guess my answer to your question is that we’ve taken very different approaches by necessity. And because of that it’s difficult to sort of cross-supply lessons from one to the other. I think maybe you could learn some lessons in terms of what features have worked and what features have not worked. You know, I would love to see. Yeah, on the other hand we’ve sort of separately implemented a lot of the same features things like documentation and hover and auto-complete… I mean I guess these are just the sort of expected set of interactions that you have now.
AL: One thing I noticed when I played with CodeWorld which I really liked as a feature but is probably going back to this non-currying decision to some extent that if you’re writing a function, it shows you which argument you’re currently in so it is sort of a type it has.
CS: Um, yeah. Um, Oh yeah, yeah, that was.
AL: Type signature and it highlights the parts of the tuple essentially where you’re just navigating.
CS: Yes, and that does definitely depend on being able to assume that every function application involves a tuple of arguments and that way you can just sort of scan for commas and so forth and figure out where you know one argument you’re in.
AL: Okay, so the other question that came up while you were talking about your new job was that you said it enables you to upstream more stuff to Haskell in general and I was wondering another thing that I’ve seen you work on and talk about recently is HMock. Does this also somehow come out of your new job or does it predate it?
CS: Yeah, no this predates my new job. So HMock actually came from prior to my working on Haskell professionally. It came from a thought experiment and this thought experiment was what are the things that I do as you know at the time primarily a Java and C++ programmer professionally what are the things that we do on a regular basis on a sort of daily basis that I just don’t know how to do in Haskell and so one big answer there comes with you know, writing unit tests using mocks is and I think people can have different opinions on the wisdom of writing unit tests using mocks ah, sometimes if you can avoid it’s good to avoid on the other hand it’s really helpful sometimes in isolating specific properties and saying I want to write a test that this specific glue layer works as it’s intended to do and those tests can really help with your level of confidence in a piece of software. But I didn’t really have a great answer for how I would do something like this in Haskell and I think we should be clear about what a mock is because I think you know a what there’s a set of terminology that’s maybe not widely understood or not universal among software engineers about what a mock is versus what a fake is. So if you have a fake that means you know say I’m interacting with the file system maybe if I don’t want to actually read and write files then I will maintain some sort of in-memory implementation of a file system and then test that my code works properly against that. And that’s what I would call a fake. What I would call a mock is if I’ve got something which has really no implementation at all and I really just want to say hey expect this function to be called with these arguments and this should be the result and if this function is called with those arguments that should be the result. So what happens is essentially the test says you know I’ve got this layer of abstraction sitting on top of that layer of abstraction I’m going to sort of stub out the bottom layer of abstraction by just saying hey if this function is called with these arguments that’s the answer and then I’m going to test the the top layer of abstraction works correctly with respect to that sort of stubbed out implementation and it sort of isolates your testing at a certain layer. So it’s not a you know mock in that sense the way I’m using the words are not fakes. They are something different. They they are.. There’s no implementation at all except that I say when you call this function I want you to you know when I call this function I want to get this answer.
JB: So to make that bit more I mean. I’m actually quite ignorant of all these testing strategies I usually do Click check and I’m happy so actually like this prevendure to mock testing here. So taking your description to something concrete would that for example, be I’m testing the sort function and I’m mocking the comparison on the elements assuming the sort function is polymorphic?
CS: Oh yeah yeah you might think about it that way. Although I think I would only apply mocks for effectful code. Um, so maybe you could mock out non effectful code that’s an interesting. You certainly can’t do it with HMock with my library so for the most part I think you’re looking at effectful code is where you want to apply these box. So a great example I think that I’ve sort of run with in presenting HMock is, suppose I’m implementing a chat bot and I want to you know something like lambda bot or something like this and I want to you know, have it interact with people over a chat channel and then I want it to do the right things in response to the right incoming chat messages. Well, so I I essentially my chat bot has this function that says pull for incoming chat messages and it just sends a you know a channel identifier and it gets back a list of you know pairs of username and chat message or something like this. What I want to do is I want to say well the first time that this poll for incoming chat messages thing happens then it should get back, you know Alice says this and Bob says this and Chelsea says that right and the second time it gets called it should get back something different so you know, Edwards says this and Fernando says that or something like that and then I want to assert that this chat bot is sending the right messages back out to the chat channel in response to the incoming messages aimed at it.
JB: So where do we have the two layers of abstraction here in this example
CS: Right? So right? So the two layers of abstraction here are you’ve got one abstraction layer which is you know interacting with the chat channel so pull for messages send a message things like that and you’ve got this other higher level of abstraction which is like being a chatbot. And so what I want to do is I want to test the chatbot functionality but I don’t want to test the interacting with the channel functionality that I’m sort of assuming is correct and so wha I want to do is not have an implementation of of a chat channel. But rather to just be able to say like the first time you pull for messages you’ll get these results and the second time you get those results and then I also want to be able to say things like and I expect that you will so you know call send chat that will be executed this many times with these arguments, and things like that. So I’m essentially like I’m talking about what’s happening at the chat channel API, but then the thing that’s actually interacting with that is the code that sits on top of this chat channel API that implements the chat Bot. And so I want to be able to sort of set up here’s what’s going to happen at the chat channel API layer and then I want to run my chat bot and assert that what really happens matches those expectations and so go ahead.
JB: Okay.
AL: So, if you’re designing anything like a testing library I imagine it’s very hard to do if you don’t have anything to test that library on or try that library on out yourself. So what have you tried it on so far? Have you been putting it to use in CodeWorld or have you been putting it to use in some other hazard project.
CS: I have not been getting it to use in CodeWorld. No that’s an interesting idea and one that and then maybe I should look into. Although I think it’s not a great fit for CodeWorld because the CodeWorld API tries to be so concrete and declarative. You know there’s no type classes and the whole design of HMock depends on well I’m going to use type classes in this sort of traditional mtl way to abstract over my effects. Um, and so.
AL: Right, but I was not so much thinking about sort of the just the surface CodeWorld language but the actual CodeWorld implementation right?
CS: Yeah I think there are places within the implementation of CodeWorld that I could use HMock in order to do some more testing. So I think that’s true. It’s also difficult to test CodeWorld because a lot of CodeWorld is designed to run on top of GHCJS. Ah though I suppose that’s not true of all of it. But I haven’t used it there. To be honest, what I have done is I’ve applied sort of my expectations from many years of writing tests using mocks in Java and C++ and I’d sort of know what I expect to have in a mock library that I’m happy with and then it’s just been about doing that. The one really interesting application of HMock which I did not start out expecting was that I use HMock to test itself and specifically there’s a lot of template Haskell code in HMock and I’d like to test that template Haskell code but testing template Haskell code is challenging in some ways right? Ah, for one thing it’s hard to test the failure cases because they result in your tests not building. Well another thing that I ran into is that I’m a really big fan of HPC for measuring test coverage, but HPC does not capture the code that runs only at build time via Template Haskell and so to sort of address these these kinds of problems actually set up a sort of an HMock API around the quasi type class that’s used in Template Haskell to actually run the code that generates Haskell code at build time so that I can run it at runtime instead of at build time and so within the testing of HMock it actually mocks itself in it mocks the or it mocks the quasi type class in order to run its own Template Haskell generation code and sort of make assertions about what happens there. It’s not typical mock testing. But I think that’s one of the really powerful things about having a library like HMock is in this case I’m not really so I wouldn’t even say I’m so much testing like a mock so much as I would say I’m actually setting up the expectations by running some Template Haskell code in anticipation at build time, and then remembering the results and then setting those expectations so that when that same Template Haskell code is executed at runtime it will give the same results, and then I can therefore like sort of stub out the Template Haskell so that I can run it at test time. And it will have the same effects that it had when it ran at build time call the same code so that I’ll measure the same code coverage and so on as what happens when running this code at build time, so and then I can assert that, for example, it reports error messages when I want it to. And I can you know I can look at my HPC results and see oh look I didn’t look at a lot of this this piece of depth of Haskell code right? here is never run by my test cases. So I’ve got a bunch of untested code right here which I should figure out why this code exists and write test cases that exercise it and this was actually this was actually really helpful. So I think like this is the one place where I’ve seen HMock really shine is in testing HMock! I found bugs and places where errors were not handled correctly. And places where you know I needed to test new cases things like this by using standard Haskell test tooling but in conjunction with using Mock on template haskell.
AL: I just want to briefly interject. Perhaps it is just my misconception but I agree with you that HPC which is Haskell program coverage is great and I think it is actually very much underused or even like little known for some reason I mean I can only say that it is part of GHC. You don’t need to do anything everybody should use it. It’s a fantastic way of getting extra information about program coverage and, yeah I mean it just is a great little thing that so many people don’t seem to know I don’t know why.
CS: Yeah I agree. Yeah I find it magical in some sense like I’ve used test coverage in other programming languages but in an eager evaluation programming language test coverage just tells you whether code was executed not whether you even looked at the result. But in Haskell and with HPC test coverage I’ve seen people just stare in amazement at the result of HPC because it will highlight the specific portion of their error message string that they did not look at or validate in their desk.
AL: Right? Also because you get expression level granularity whereas in many other languages you only get sort of line level granularity. So yeah, it’s really very nice.
CS: Yeah yeah I mean that like the granularity and also just the lazy evaluation lets you know when you know maybe okay, there was some code that filled in this value. But if I never looked at the value if no other behaviour depended on the value that filled it in then that’s still untested code. So I yeah I agree I saw HPC for the first time when I was working on the snap framework with Gregory Collins and Doug Beardsley back in in 2011 and they you know had tooling set up so that you could look go look in there on the snap framework website and you could sort of see the HPC coverage for each module by unit tests and you know so I contributed some you know xml and html parsing and rendering code to that’s used by heist there and saw that tool for the first time there and it was great. Yeah I’ve been a big fan of it ever since I think it’s maybe not the best maintained piece of code but it works.
JB: Yeah I wish we had that for GHC just last week I was wondering about a certain case branch in GHC which I thought is probably dead code and what I did I replaced it with error and made a draft pull request against GHC and ran CI on it to see if it fails and indeed this code path was not taken. If I had had a HPC report created maybe automatically on the CI system could just say okay this code is dead and it’s not exercised by the test suite. So probably my theory that it’s actually not reachable is correct. So it’s also useful for that.
AL: So HMock is basically ready for production use you would say? I mean so.
CS: Yeah I think there are definitely people using it and I’ve heard good things from people who are using it. I hope to become a person who’s using it at some point in the future. But I think I have to wait for the need to arise I’m not going to force it into a hole where it doesn’t fit. Um. So but I think it’s being used successfully by Haskell programmers. One thing that I would love to do more with is to sort of generalize it a bit and so I’ve resisted declaring anything to be one dot oh release yet. Because I know that there will be breaking API changes needed if I were to generate generalize HMock to other sort of effect systems. You know I would love to have you be able to mock out, you know, servant API calls very easily or Hazel or various effect system libraries like polysemy. Things like this. Currently, it’s very specialized to mtl style type classes which I think are the the low friction way to abstract over effects but are not necessarily what everyone is doing so I think I have some ideas for how this would work and I hope to get back to this and be able to sort of generalize this to the point where I can really say like hey no matter what no matter how you’ve designed your effect systems and how you’ve abstracted over those effects that you can you know you can use HMock somehow to mock those effects using you know, various different plugins or various different. You know HMock subsidiary libraries or something like that. So I haven’t declared it to be stable for that reason I’m still thinking about that problem and I’ve reached out and and talked to people about this question. I would love to have it generalized enough that it’s just sort of the default way to mock things in Haskell. Right now I think it’s a good default for mocking mtl style type classes in Haskell but there are a lot more things that should be mockable.
JB: Okay, so to summarise Haskell the language you don’t want to make any big changes Haskell the library ecosystem obviously had a big missing piece in terms of a mock library and we have there. So let’s talk about Haskell the community and ecosystem because it’s also a place where you’ve been quite active in the last month maybe and in involvement with Haskell Foundation. So, tell us a little bit more what you’re doing there at the moment.
CS: Yeah I think I really am excited by the Haskell Foundation which is now coming up on getting close to a year old if you count from the date of announcement maybe less if you count from the date of it’s having full-time employees. But and think it’s been really exciting to me to look at the Haskell Foundation as a mechanism for getting a lot of things done within the Haskell community and by getting things done I just mean like anything that kind of needs to get done. I feel like the foundation has a bit of a track record at this point and at least is well placed to be an encouraging force to make improvements to Haskell and so I’ve been pretty excited about this and I’m made a decision to become a sort of financial contributor to the Haskell Foundation back over the summer and so I become that so I’m contributing to the Haskell Foundation and that sort of makes the question a little bit more imperative to me of you know how can the Haskell Foundation be something that I feel as good as possible about contributing to and about supporting and so there’s been a lot of conversations going on about this recently about you know what should the Haskell Foundation be doing that makes it a really attractive or even more attractive target for someone who says you know hey I want to support Haskell you know as a community just all of Haskell you know, right now there you know there are ways to support the Haskell.org committee which you know hosts the haskell.org website and some infrastructure things. There are ways to support GHC development. There are you know ways to support hls they’ve sort of got their own you know donation system set up, there are a million people not a million, maybe a dozen people who have set up, github sponsorships or patreon pages to support their work on the Haskell ecosystem. But as someone who says like I want to support Haskell it’s not clear to me that either of these is necessarily like the choice for saying I just want to you know help out the Haskell community and I think the Haskell Foundation has the potential to be that choice.
JB: Simply by distributing the funds into these channels you’ve just mentioned or do more than just.
CS: Well, that’s a good question right? That’s a good question I think it’s definitely more than just distributing funds. You know part of it is the Haskell Foundation is you know, well placed to build excitement and sort of bring the community together around projects. And I think you saw that with like the text utf8 project where, here’s this this change that a lot of people have said is sort of a no-brainer for a long time but it’s at the same time when you get into the weeds of it you just lose momentum and little technical issues come up and there’s not, at at some point you need the marketing push behind a project in order to push it through to success.
JB: But is that I mean I really am really fond of this change but is this really the poster child example for a project that we need to ask the Foundation for I would.
CS: Ah, you know I don’t know I think I see it as a success story. Um, but I Ido think that there are different roles that the Haskell Foundation can play. I am really excited about one of the proposals that’s sort of out there that I’ve been working on is to have a grant program to take contributions to the Haskell Foundation and find those areas within the Haskell Foundation where it could do some good and just say look if you are you know a thousand dollars short of being able to spend a month contributing toward a Haskell project you know and doing some open source work for a while then we want to be there with that thousand dollars to say well we’ll help you meet the gap. We’ll help you know, match that need. And you essentially take someone it’s certainly not going to pay for full time developers to work on Haskell it’s more about finding accidents of opportunity right? It’s about finding cases where someone’s what someone is saying what I really want to be doing is working on improving Haskell’s GHC to Javascript cross compilation support via asterius or GHCJS or something like that and this is just an example you know, maybe somebody’s like really all set for doing that and they’re like well and I’d love to be doing this but you know I just I still need to pay my rent. Um, you know, not that I want to be paid a competitive software engineering salary but I need to to be able to you know, eat dinner while I do it and you know they should be able to at least ask the question of hey does the Haskell community want to support me in doing this work. So that’s one example I think there are other examples of how we how the the foundation could be more worth supporting as well. You know in some senses those go toward maybe we maybe the foundation should support events more than people or maybe the foundation should support, you know more outreach to newcomers or something like that. Yeah it so I think different people have different kinds of of ideas about how this works out, but it’s a good conversation to be having. So I’ll throw on a plug about this conversation which is that that Matthias Tope and I are thought I don’t know how to pronounce his name I hope I got it right? But we are actually looking for people to join a committee which we call the foundation task force to help us sort of talk through the question of what these kinds of efforts might look like for the Haskell Foundation and we would definitely love to hear from people who have ideas and you know are excited about other ways that the foundation could be supportive to the community.
AL: Yeah, this one is really exciting! I hope you win a lot of extra people for this effort and succeed in effecting some changes. So I mean I want to perhaps as a final topic, to talk about another way in which I’ve recently seen you engage with the community and that is that you I think recently started I think it’s called a co-hack a virtual co-hack beat up.
CS: Yeah, yeah, that’s actually got a bit of history behind it. So when I was living in New York working for Google’s education group there the New York Haskell user group started these meetings and I ran these with the New York Haskell user group as a sort of in-person meeting. Microsoft actually donated some office space for us on weekends and we’d get together a group of people and we’d sit around and we’d just work on Haskell projects together and you know it’d be everything from you know, somebody coming in to say hey I want to understand monads by implementing monads in python and so you know they work on that or you know a lot of times we get people coming in with some rather heavy like category theory ideas they wanted to work on or some people are just saying hey I want to work Haskell you know, let me start with a Haskell project and does anyone want to help me out set it out and we would spend 4 or 5 hours in this conference room and just people kind of talking to each other and working together on things order pizza you know, but it was a lot of fun. Actually the idea there came from Daniel Cartwright also known as chesseye started something similar to this in Atlanta. Which I attended here in Atlanta and then sort of expand that expanded to the co-hack in New York. After covid obviously you know Microsoft Research stopped donating in-person meeting space since everyone’s virtual now and so that’s sort of shut down. So lately I’ve just been missing that and so I’ve started to try to organize over zoom as you know the the best we can get at reproducing this kind of social, you know low-key social meeting of Haskellers to get together to work on projects together to meet each other so you know every month we get together and you know assign people to random breakout rooms to meet other people there at the meeting and then you know various people give pitches about things that they’d like to work on work that they’d like to do stuff that they’d like to learn etc and then we just break up into groups and work together on things for a few hours. It’s been a lot of fun.
AL: That sounds really cool. So I guess if you’re organizing something like this and advertise it globally then you can end up with either 3 participants or 500.
CS: Yeah, it’s been closer to maybe a dozen or so than 500 recently but we’ll see which structure it goes. I mean honestly if four or five people show up and there are four or five people who have fun together then it’s worth doing.
AL: So do you pick some theme or topic for these sessions in advance or is it completely self-organizing essentially.
CS: It is completely self-organizing. We do have a period of time set up for anyone who wants to give a pitch. So if someone shows up with their own project and says I would like to you know work on this particular project during this time period and I’d like some help some pair programming or you know or someone to jump in and work on it with me then they could give a pitch and we’ll create a breakout room for them so people can come join their rooms and you know they could talk to each other. But aside from that it’s just whatever people want to do.
AL: And is the tooling up for this kind of stuff? I mean how do people usually do this? Is one person sharing their screen and the others are looking at it or are you using some official visuals your live share?
CS: Yeah, within each of the rooms I think it’s typical that maybe one person is sharing a screen and other people are sort of following along there. But there’s no real rule about tooling. So if someone wanted to use the like code, the VSCode extension
AL: CodeWorld
CS: That well I mean if someone wanted to use CodeWorld, they could do that. If someone wanted to there is like a vs-code extension that lets like multiple people work in the same code file at the same time I’ve forgotten what it’s called to be honest, but if someone wanted to push for that they could do that too. But it’s all organized through zoom and so sort of screen sharing is the easy thing to do you just click the button saying share my screen and then anyone in the room can see the screen.
JB: So that sounds cool which channels do I have to watch to and learn about the next one?
CS: Oh, I try to announce it on like Twitter, haskell-cafe, haskell-reddit, haskell-discourse. Um, so.
JB: Okay, so I have to try really hard to not notice the next announcement…
CS: I think that probably you probably would have to try pretty hard to not notice. Yeah, so that would yeah I guess the podcast will be delayed so it does not good to say that this Saturday is the next one because it will be after this Saturday when the podcast gets released.
JB: Well cool! I think that sounds exciting. Maybe I’ll see the two of you at one of these future co-hacks! I hope I see many of our listeners at these future co-hacks and I think with this I’d like to thank Chris for this very interesting hour of conversation.
CS: Thank you.
JB: And I hope to have all the listeners again at one of future episodes and until then have a good time.
CS: Thanks!
