[00:00:00] Joel: Hi, Angie.
[00:00:01] Angie Jones: Hey, Joel, how's
[00:00:02] Joel: It's going really well. I'm excited to talk to you again. I know we've chatted in the past and done a podcast episode, but a lot has changed in the meantime, you've done a lot of new things. I'm up to new things and I thought we could come together and kind of talk about that. And mostly I'm interested in talking about courses and learning platforms
[00:00:17] Angie Jones: it going?
[00:00:18] Joel: sort of thing,
[00:00:18] Angie Jones: Yeah.
[00:00:19] Joel: And you've built something really nice with testing automation, U but first I wanted to kind of, break the ice a little bit and I was wondering. When you sit down and you are personally learning something new and you have a new space that's open and you're gonna sit down and learn something complex.
Learning complex subjects
[00:00:33] Joel: What's your approach and how do you like to learn new things?
[00:00:36] Angie Jones: Yeah, I really like When I'm giving examples, you know, concrete information and you would be surprised, well, maybe not you, but some people may be surprised just how rare that is. So when I'm researching new topics and things like that, there's a lot of kind of information. And everyone tries to sound really smart and it comes off like very academic versus, you know, practical.
[00:01:02] So what I do is one is. Find like someone that I can learn from someone I can vibe with. And sometimes I have to go through a couple of courses before I find that just recently in something I'm learning I can't really find one person that I vibe with. So I'm learning from a couple of different people and then trying to put it together so that I understand better.
[00:01:27] Joel: yeah, multiple sources. And I don't know if you've experienced this and you kind of touched on it where the examples are, where, or it feels like it's academic. I feel like a lot of times people are actually teaching theory something that
[00:01:37] Done in real life, which there's a huge difference.
[00:01:39] Right? Theory's fine. But. It's a big difference to me.
A frustrating lack of practical examples
[00:01:43] Angie Jones: I know, and then everyone kind of repeats what they learned from the other person. So like last night I was looking for this definition of this concept in Java
[00:01:52] Every resource I found gave like the exact same sentence as that definition. And none of it told me the what is this used for? And you know, why would I want to use it?
[00:02:06] It was. Like, oh, this is an interface that does X, Y, Z. And it's huh?
[00:02:10] Joel: Why do you think that happens? What causes that and causes people to, to teach and just kind of recycle material? Yeah.
[00:02:16] Angie Jones: that's a really good question. And I think maybe not putting themselves in the learner's shoes. Right. And so if I look up, well, how do I best describe this concept? And there's five or. Authors who have all described it exactly the same, then it's natural. I say, okay, well that is right, right. This has been confirmed by five or six experts.
[00:02:41] So I'll just say the same thing. And I actually need to teach that concept. So this is a really good example. I refuse to teach it that same way. Joel I'm so I'm trying to figure out right now, like how best I'm basically in the learner's seat right now. Like I. Stand this in my head, but trying to communicate the concept out w is gonna be kind of challenging.
Why before the what
[00:03:05] Angie Jones: So I'm trying to put myself in. Okay. What if I knew nothing about this? What I would most wanna know is not. What it is, but why I want it what problem does it solve? And then you can tell me like the academic little tidbits about it, but for right now, I just need to know the why.
[00:03:25] Joel: I think that you touched on something that's extremely important too. Is that what is the problem being solved here? Like why even bother doing this and what am I gonna be able to do when I get done is like a critical aspect that seems to get missed a lot.
[00:03:37] Angie Jones: I know, because in these tech courses, essentially what we're giving people are tools, right? And so you can teach me about a hammer, but if you don't tell me, when am I supposed to use the hammer, then what good is that right? Maybe I start using it all the time for everything or, you know, so I think that's.
[00:03:56] More important than the, what is the win. And you know, then I know okay, here are all of my tools and I know what problems each one of them solves. So I know which one to grab when I face a certain problem.
Differences between research and personal learning
[00:04:08] Joel: When you sit down and you know, as an and you say, you know, like you're trying to capture that beginner's mind, is your research process different when you're designing, learning material than when you're learning something yourself? Or is there some overlap there?
[00:04:20] Yeah.
[00:04:20] Angie Jones: That's a good question. It might be a little bit. Different. Maybe maybe there's some overlap, but a little bit of difference. Right. I have a lot of foundational knowledge. And so I can draw on that a bit. And so I try not to assume that my audience will have that same sort of knowledge.
[00:04:37] Right. So I've been in tech for almost 20 years at this point. I might can skip ahead a little bit, or just give me the, you know, the TDLR of this thing. Versus when I'm teaching, I try to go a little bit more into detail.
[00:04:53] Joel: Where do you draw the line in terms of the assumptions? Because
Where to draw the line with assumed knowledge
[00:04:55] Joel: There's, you know, like we, how many times are we going to explain what an array is though, somebody's gonna need that. And they might need to remediate and understand that. Like how far do you go back and what are the assumptions you're making when you're trying to teach.
[00:05:07] Angie Jones: Yeah, that's a good question. I think it depends on the topic. So if
[00:05:10] I'm talking about something like maybe loops, for example maybe that's not a good example. I was gonna say, well, then I can make an assumption that you've already learned or raise or something like that, but you get what I'm saying.
[00:05:22] If I. If I'm teaching something that's applicable to another concept, I might just do
[00:05:28] Joel: Yeah.
[00:05:29] Angie Jones: liner kind of refresh your memory sort of thing, but not go into detail about it.
Designing an engaging course
[00:05:35] Joel: When you're doing this. And I can't agree more that the examples are one of the most important things. Like something that I can get into, I can dig in and feels pretty real. do you think about actually designing those exercises when you're creating a course?
[00:05:47] Angie Jones: Yeah, that's the best part of my courses, I think is that I give realistic examples. So there's no food bar in my courses. I try my very best not to just, you know, give generic Details and variables, but I say let's solve a problem. And I take quite a bit of time trying to figure out good fun problems to solve that will really demonstrate the topics that I'm teaching.
[00:06:15] I draw from everything like I'm notorious for putting like a little small game in my courses. You know, so let's build this game or whatever. And it could be like really simple. It could be a number guessing game or a dice game or a card game. You know, something simple that won't take too long, but it's fun.
[00:06:33] It's something you can get behind. You understand it from a practical standpoint and now you just need to apply the code to it, you know?
[00:06:41] Joel: Yeah. How important is fun in, in learning? I think this is something that, that we debate and even the definition of fun can be kind of fuzzy, but like how important is it to have fun while you're learning or teaching?
[00:06:50] Angie Jones: Yeah, I think it's extremely important. Because we're all so busy and it's very rare for someone to sit down in one sitting and take your course, right. They have to be motivated to come back and if it's dry and boring and you know, the examples don't resonate, then they're not gonna come back.
[00:07:11] Right. Unless they just. Absolutely need it, but they're suffering through, even if they absolutely need it, they might go find someone else to get it from hope. That's a little bit more exciting. So I definitely try to add fun. I add little jokes in my courses. Like I said, the projects are really fun, so let's build a game.
[00:07:29] I have one, let's build a lottery ticket generator and you know, when you play the lottery and you win, don't forget. Teacher, Angie, you know,
[00:07:37] Joel: Right.
[00:07:38] Angie Jones: but I've actually had someone, you know, generate their numbers and they won a couple of bucks on a scratch off or something. Not a scratch off, but a lot ticket.
[00:07:48] Yeah, I think that it is really important that you make it engaging, just like anything else. If you're doing some sort of presentation or talk, you know, you want to be engaging same goals for a course.
[00:08:00] Joel: There's a I also think that the other side of that is that there's this disconnect in training and like the real life processes of developing software in the enterprise. And I'm wondering, is there a way to bring those two together and still maintain that fund, but delivered or is that just kind of covered by the example space and people are able to transfer that over.
[00:08:19] Angie Jones: You know, I feel like that's a major gap project based stuff in general. So a lot of the courses I see are very academic and they give you like bullet points of the information. Here's a topic. Here's the rules about that specific concept.
[00:08:36] Joel: like the
[00:08:37] Angie Jones: And. Yeah. And then here's a a very generic example. It's not specific to a real problem.
[00:08:43] I just wanna show you, you know, the syntax of something, right. And what we are finding. And I truly believe this is why. It's so difficult for a lot of people who are trying to enter into the tech space to find their first job. And, you know, they've done all, we've told them they've taken the courses. We gave them a list.
[00:09:06] Here's what you need to know. They did all of that. They haven't built anything. They don't know how to apply what they've learned to real enterprise type problems, you know? I really encourage instructors to add You know, little projects to their courses. I do that with, in some of my courses when I'm allowed to every platform, won't allow me to do this, but when I'm allowed to, I try to add you know, optional exercise to every single chapter that I teach so that you can.
[00:09:36] You listen to me, I go through an actual real world example as I'm teaching you. And then I give you another real world example, a problem to solve with the information that I've just taught you. And I think that's.
[00:09:51] Joel: right?
[00:09:51] Angie Jones: Yeah, so you can do it yourself without me. Like it's easy to follow along when I'm showing you, but I want you to be able to do it yourself and you might have to replay the VI the video or look through the transcripts.
[00:10:02] That's fine. Right. That's what we do as developers, but I think applying that knowledge is the key and many courses are missing that. And that's why these folks are having a hard time on the job. So I think we could still do the fun projects. Like I said, I, mine might be a game, but it's really making you think and exercise these concepts.
[00:10:25] Joel: We talked about this on Twitter too where people are building portfolios and they're not building portfolios that are reflective of the jobs that they want to get. And often that's tough hurdle that you have to get past, cuz you have to build something real to get a real job.
[00:10:37] I think at the end of the day,
[00:10:39] Angie Jones: Yep. Yep. And they are really struggling with
[00:10:42] Joel: Mm, One of, one of the interesting disconnects that I've seen. And I don't know if you've noticed this at well is when we're working, it's like this thing where you have product and engineering and marketing and you have all like the
[00:10:52] Angie Jones: that.
[00:10:53] Joel: In there changing stuff on you.
Simulating a work environment
[00:10:54] Joel: And have you ever thought about how you might simulate that and kind of a training environment to, to actually give people a chance?
[00:11:00] Angie Jones: No, I haven't. And that's a really good observation. When we're teaching courses, it's very much an isolated experience, right? This is very independent. You're studying on your own, even if you think about. School settings, you get penalized. If you know, Google and find a solution, or if you work with someone else and it's not a group project, you know,
[00:11:23] Joel: Yep.
[00:11:23] Angie Jones: I think that's MIS and I didn't learn about a lot of that stuff, Joel, until I went to do my master's degree,
[00:11:30] Joel: Yeah.
[00:11:31] Angie Jones: is when they taught about software engineering, not programming, but software engineering and talked about all of those different roles that come into.
[00:11:40] Joel: Yeah, I know that I, there's a couple interesting novels that have been written in a narrative form that try to show people that, and I've heard like Microsoft does their internal in the actually hire actors
[00:11:49] Angie Jones: Oh,
[00:11:50] Joel: kind of simulate the work environment. And I think that's an amazing concept, but I just haven't really even seen that.
[00:11:55] And even in my own teaching, it's always Here you are. We're talking about this, it's in isolation, you're working by yourself. You know, you have some assets, but it's not you know, you're collaborating with a designer, which is a lot different than being, you know, handed a design or having to create your own.
Testing content
[00:12:07] Joel: Have you found any way to test your learning content when you're creating a course? Do you, are you able to test that with people or do you just kind of, create it for the event or the platform that you're building it for and kind of test it in production,
[00:12:18] Angie Jones: so the ones on TA U I didn't test those. It, you know, I basically just. Let it ride. But on some other platforms like LinkedIn learning is another one where I have courses and they have a set of testers that take the course before it's released. And I thought that was really neat and helpful.
[00:12:40] And, you know, they'll call out, you know, continuity issues. And this isn't really clear. You kind of lost me here and there. So I that's been very beneficial.
[00:12:51] Joel: Would that actually affect the course? Would you go back and fix those things or improve based on
[00:12:55] Angie Jones: Yeah, I do. I go back and I may have to record another little snippet clarifying point or, you know, something like that. It's a pain, but you know, if I feel like it's really necessary, then yeah, I'll do it.
[00:13:07] Joel: Did, when you were building testing automation university, is that something that y'all built from scratch or did you use framework or platform for
[00:13:14] Angie Jones: Yeah, we built that from scratch. I actually hired a contractor to, to build that out. I didn't wanna build it that way, but we decided to go that route. Just to have a more flexible platform that we can add, whatever features we want it to. But if I were to do this all over again, I think I might go with a what do they call learning management system?
[00:13:37] Joel: okay. a kind of a formal enterprise learning management system
[00:13:41] Angie Jones: Yeah, I hear those are pricey though, but
[00:13:44] Joel: That's true.
[00:13:45] Angie Jones: yeah, but that I would rather go with that. It was a, it is a lot of work to deploy just a single course, right? Like I'm literally in here I had like editors and I'm, you know, converting transcripts to mark down and all of this stuff, it was quite the effort.
[00:14:02] It would take me like a day or two to deploy a course.
[00:14:05] Joel: If you were designing or you could snap your fingers and have the ultimate course platform at your disposal, what are some of the features that you'd have to have that you'd want to have to make those problems go away and deliver? better learning experience.
[00:14:18] Angie Jones: Let's see. I would, I definitely need video. Transcripts quizzes. I really like gamification type things, so points. We added some things on test automation, university leaderboard and stuff like that. So for those who like competition that's really fun as well. I like a lot of the elements that Josh has.
[00:14:40] What's Josh's last name? Yes
[00:14:42] Joel: His
[00:14:43] Angie Jones: in his CSS course, like the interactive elements there. I would totally steal that sort of thing.
[00:14:50] Joel: It's inspiring what he's done and just that ability to go through and. Touch the screen and have something happen and just ingrains it right in your brain. And I love it.
[00:14:59] Angie Jones: yeah. That's the kind of stuff that makes you wanna come back and take the next lesson, you know?
[00:15:06] Joel: I think gamification kind of gets a bad rap. Like people think about that and they think of, you know, like tokens or something flying up in your face or whatever, but how has that improved, the learner experience and people that have interacted with testing automation.
[00:15:18] Angie Jones: It's been wonderful, right? So they have quizzes and you get points for the quizzes. And like I say, we have a leaderboard, we have a TA U top 100. And so they're the people who are in the top 100 are pretty competitive. And so they end up taking. Any, and every course that we put out because they wanna get their points up and keep their slots.
[00:15:41] Right.
[00:15:41] Joel: . Yeah.
[00:15:42] Angie Jones: but I just think of wow. Think about the sort of tool belt that this person now has. Like we have our courses on test automation, university are split amongst like UI and API and mobile apps. And so you're learning like all of these different lanes And everything about them, right?
[00:16:02] How powerful is that? But yeah, they really enjoy it. We give little cute badges upon completion. So people share those and they share their certificates and things like that. So it's not a, in your face kind of thing, but it's still like really fun.
[00:16:19] Joel: So I was talking to Aman Zabi who does the terrain.com is their course platform that they built. I want to run this idea by you. Cause I think it's one of the most interesting gamification techniques and kind of maybe aligns with your current interests, but they've taken the and they give participation points to the learners. And at the end of the month they take that pool and then they actually profit share with the learners. So the most, the learners that participate the. End up actually getting they pay out in crypto and cash and give people, you know, like actual money for participating and taking the
[00:16:48] And I'd never heard of that before. And I think it's like one of the coolest kind of modern web three ways to, to incentivize the people, to do the activities of learning that you're trying to get them to accomplish.
[00:16:57] Angie Jones: That's.
[00:16:57] Joel: that?
[00:16:57] Angie Jones: I think that's really neat. That's really neat. I wonder how that's working for them. Some folks are just not motivated by money though, you know?
[00:17:05] Joel: Well, it's like an ex extrinsic motivation and I think
[00:17:08] Angie Jones: Yeah.
[00:17:09] Joel: Teachers assistance and people that are active in community and helping others more than
[00:17:13] Like act, you know, like actively learning. And I think that's part of it too, but it's like, how
[00:17:17] Angie Jones: Yeah.
[00:17:17] Joel: people in the community answering You know, like being
[00:17:19] Angie Jones: Ah
[00:17:20] Joel: and really helping, then fairly compensate then too. Cause that's work. Right? So
[00:17:24] That they're going
[00:17:25] Angie Jones: No, I really like that, but, and those are the people you kind of need, you need those community members that are lending a hand and things like that. So it's really nice to recognize and compensate them a bit.
[00:17:36] Joel: For me it's tricky too. Cause I don't want to, don't want to exploit the community. I don't want them to
[00:17:40] Literally doing labor for us at commercial business and you know, not getting compensated for it fairly. So I thought that was just a clever way to, to do that and kind of help motivate people and, you know, like share, share the success of the platform with the people that are using. The
[00:17:55] Angie Jones: absolutely.
[00:17:55] Joel: You've built a learning platform and you had help and somebody built that platform for you or worked on
Balancing the platform development and content workloads
[00:18:01] Joel: But at some point you are having to think about the platform and the delivery and the learning experience. And I'm wondering how do you platform and creation and design of, of the delivery mechanism
[00:18:11] Angie Jones: Ah.
[00:18:12] Joel: creation of the content that people are going to interact with the courses.
[00:18:15] Angie Jones: Yeah, fortunately in the beginning we had to focus on both The platform itself, but then also we really needed to put courses out. So I kind of tag teamed this with another coworker and had that coworker focus on the platform stuff while I focused on getting the courses out once the platform matured.
[00:18:39] Then it was less work. It was more just maintenance there. Here and there, we might have request for a new feature or something, but most of the attention was needed on the courses.
[00:18:49] Joel: What makes a really good course? We've talked about it a little bit, but just if you're gonna describe the
[00:18:53] Angie Jones: Yeah.
[00:18:54] Joel: What, how does it feel and what goes into a really good course?
What makes up a good course
[00:18:57] Angie Jones: Okay. A really good course. I think one is a good needed topic, right? Two is the length of the course. So anything that's too long, I found people don't finish them.
[00:19:09] Joel: Yeah, that's
[00:19:10] Angie Jones: That's true. Right. They don't, they won't finish them. So you have to be mindful even if it needs to be like a part one and a part two or something like that.
[00:19:19] But the length of that course I found the happiest spot is about an hour. What have you found? Joel?
[00:19:27] Joel: It really kind of depends. And you gotta break it down and people's attention, Wayne. So if it's a live workshop, four hours, like a
[00:19:32] Is really after that. Like people just glaze over. And then when you're trying to get people to, to go through a course, I feel like, you know, you have to break it down into chunks at the end
[00:19:41] Might be a longer course. It might be a four hour course, but let's break it down into chunk. So at each module, each hour increment, you're getting like some value and you're achieving something so that, you know, like you've achieved it. You can stop. Come back later.
[00:19:54] Angie Jones: Yeah. Yeah. And then also another element for a good course, a of again, is the engagement part. I think that's what really keeps them, cuz you're not just listening to me. I do kind of code alone style. So you can watch me if you wanted to, but I encourage people to like code with me. And then at the end I offer an exercise of their own.
[00:20:16] So I think that really gets them bought in. I. You know, they've now invested, they've worked on some problems. They've solved them. They feel really good about that and they continue on. So like my, I have a Java course on test automation university. That's about six hours long. So it's a really long course.
[00:20:36] And if I look at that one compared to other ones that may be three or four hours, the completion rate is much better. And I. That definitely is attributed to the the engagement factors that I've placed in there. So some of the other ones are long, but there's no exercises for folks in that
Balancing lesson content
[00:20:56] Joel: How do you balance the cuz you mentioned video and I think video is a great mechanism for delivering certain types of content, but how do you balance the text and the quizzing and that sort of thing with video
[00:21:07] Angie Jones: sort of thing. Mm-hmm,
[00:21:07] Joel: Seems to be most effective?
[00:21:09] Angie Jones: So for accessibility reasons, I hired an editor that would, or a transcriber that would. Essentially take every video and it's not just the text. Right. They actually make that until almost like an article. So they're incorporating like pictures from the video. The code snippets that the instructor is going over and embedding all of that right in the transcript.
[00:21:39] And I did that for accessibility reasons, but. I was really surprised to see how many people were using those transcripts. So some folks were using them as opposed to the video and many people. I think it was like, 60% were using it as a compliment to the video. So they would let the video play and let the instructors speak, but they were reading and following along with the transcript, which I thought was really interesting.
[00:22:10] Joel: Yeah, that's something I
[00:22:11] Angie Jones: Really? Oh, wow.
[00:22:11] Joel: I'll watch the transcript or read the transcript while listening to it on two X speed too. I like to do that to where I'm listening to it on double speed while I read, because I find videos kind of slow sometimes,
[00:22:20] Angie Jones: So if you read it, you feel like that's
[00:22:23] Joel: Yeah. So I'm kind of processing it visually and then, and it depends on what the video is, right? If somebody's actively coding or they're doing something or it's something, you know, take a, like a woodworking course. I can't, you can't read. About woodworking. It just doesn't work. But a lot of times when it's more of a either, you know, like a knowledge work type of course I'll be able to the transcript.
[00:22:40] And I, and it's honestly
[00:22:42] Angie Jones: sinking in better or what .
[00:22:42] Joel: Podcasts too. I pull up a transcript, I import them into Otter, up the transcript, listen to it on two X and watch it run through the transcript while I watch it or listen at that speed. So it's kind of engaging and gets
[00:22:53] Angie Jones: Okay. I'm gonna try that on my next, the next course I take, I'm gonna try that approach.
Completion statistics
[00:22:58] Joel: Important are completion rates in a course? Do PE is it, you know, is it really important or can people get what they need and get out or how do you put weight on the completion rates? In a course.
[00:23:08] Angie Jones: You know, When we first launched GAU, I would get questions about, you know, from my from folks working there, I would get questions about the completion rates and what can we do to improve completion rates? And it wasn't something that I even thought of like monitoring. I didn't really care about that.
[00:23:28] And I don't know if that's just me being naive or what I care about, unless there's some sort of, you know, issue oh, this the quality of the course is not good or something like that. But other than that, I just don't think people. Finish things. They just don't like, I don't finish things.
[00:23:46] Right. I take courses sometimes and it's I got what I needed. I'm done. Or this isn't as interesting as I thought it was going to be. So I'm, I screwed this topic all together kind of thing. Yeah, so I don't even, I don't put much weight on that at all.
[00:24:03] Joel: I think it's because it's like the only metric outside of following up
[00:24:07] And talking to them about their success and what the course actually did for them, which is labor intensive and difficult. So you can just take, well, we can automate this and see how many actually finished.
[00:24:16] And it's, you know, it's something to look at. I'm, it's a
[00:24:19] Debated topic, actually
[00:24:20] Angie Jones: do you, what do you think is more important? How many people started it or how many people finished it or equally as important?
[00:24:29] Joel: Only real measure of success for me in any of this is whether the students achieve the difference in their lives that they're set out to achieve in the first place. And if we're accomplishing that, we're doing a good job. It's just hard to measure that unless you are really, you know, invested in their approach.
[00:24:43] And it's particularly difficult when we're talking about self-paced courses, you know, like an egg head.
[00:24:48] Start and finish it, we get there's this cliff after the first video and literally every one of them, and you can kind of see this cliff after videos one and two and people fall off.
[00:24:55] They just, you know, this wasn't for them, the instructor isn't for them, they don't like the style, whatever. And then, you know how they go through it after that is, is varied. But often people just they're trying to solve a specific problem, a technical problem. They get their answer.
[00:25:08] They get to move on in their lives and, you know, go about their job. And that's why they watched the course in the first place, cuz they were
[00:25:13] To solve a very specific thing and it either did it or it didn't.
[00:25:15] And that, that to me is success and that's how I measure it try to into my, to my ability.
[00:25:20] Angie Jones: I like that.
[00:25:21] Joel: Angie, thank you so much for chatting with me. It's always interesting. Where can people find out what you're up to on a regular basis?
[00:25:27] Angie Jones: Yeah. Check me out. I live on Twitter at tech girl, 1 9, 8. I also have a website. Angie jones.tech.
[00:25:35] Joel: Thanks, Angie.
[00:25:37] Angie Jones: Yeah. Thank you, Joel.
[00:25:38] Joel: Cheers.
