diff --git a/flow_results/cleaned_transcripts/434-mobile-apps_clean.txt b/flow_results/cleaned_transcripts/434-mobile-apps_clean.txt new file mode 100644 index 0000000..82b9611 --- /dev/null +++ b/flow_results/cleaned_transcripts/434-mobile-apps_clean.txt @@ -0,0 +1,896 @@ +Are you building a mobile app and wondering where Python fits in in the mix? +Are you supporting others building these apps with back-end APIs written in Python? +Can you write your entire app end-to-end in Python? +Well, I have a great panel put together to discuss exactly this, and they all have a different and unique take on the options. +Welcome to Lauren Augie, Harut Bozakizan, Andreas Kuhn, Jeffrin, and Joshua. +This is Talk Python to Me, episode 434, recorded September 9th, 2023. +This is your host, Michael Kennedy. +Follow me on Mastodon, where I'm at MKennedy, and follow the podcast using at Talk Python. +Both on mastodon.org. +Keep up with the show and listen to over seven years of past episodes at Talk Python.fm. +We've started streaming most of our episodes live on YouTube. +Subscribe to our YouTube channel over at Talk Python.fm slash YouTube to get notified about upcoming shows and be part of that episode. +This episode is sponsored by Sentry. +Don't let those errors go unnoticed. +Use Sentry. +Get started at Talk Python.fm slash Sentry. +And it's also brought to you by us over at Talk Python Training. +Did you know that we have over 250 hours of Python courses? +Yeah, that's right. +Check them out at Talk Python.fm slash courses. +Hey everyone, Andreas, Harut, Lauren, Jeffrin, and Joshua, all of you, welcome to the show. +So exciting to have you here. +Thank you so much. +I'm really happy to be here. +Yeah, great to be here. +Sam, yeah, it's great to be here. +This is going to be a really fun topic. +And I think it's one that puts Python into an interesting space. +Python's really popular, as everyone listening knows. +Its growth has been super, super high since 2012. +And yet it really doesn't play a huge part in mobile apps. +Although we do have some folks representing Kivi here via their app, which is pretty awesome. +So I'm very excited about that. +But it's not the first language you might pick for writing mobile apps. +If you just say, all I care about is mobile apps, I just want to get started. +But I think there's a lot of things that Python plays a part in, right? +Backend services. +And I'm really excited to hear about the Kivi experience as well. +So there's a ton of Python people out there who either are building or want to build some kind of mobile app. +It's obviously one of the most important computing platforms. +So I think everyone's going to learn from everyone's experience here. +So I think let's just, I guess I'll go around by app. +That's the best way I can kind of know to do this. +And have you all introduce yourself. +And then actually, let's just have you introduce yourself, then we'll talk about the app. +So Andreas, we'll go around the Brady Bunch circle of pictures we got on the screen here. +Andreas, go first. +Yeah, my name is Andreas. +I live and work in Stockholm, Sweden. +And I've been working with Python for about 10 years now. +And have been working with, I used to work for an online tailor-made suit company, where we developed the platform ourselves. +And now I'm working with a membership management company, which is a Django app that we are running in the background. +Nice. +Very exciting. +Looking forward to hearing about it. +Harut. +Hey, guys, my name is Harut. +I have been working in Python for like, let's say a decade on and off. +My background is in applied and computational math. +So I did a lot of MATLAB in college, and then found my way kind of over into Python, did some data analytics, some data science. +And then for the past few years, found that building tools for people is actually what I really enjoy. +So I do a lot of work with data visualization, a lot of JavaScript. +But if there's one thing that's been consistent in the past 10 years, it's been my Python usage overall. +Awesome. +That's kind of how I started as well, building scientific tools and stuff. +Lauren. +Hi, everyone. +Yeah, my name is Lauren. +I've been programming since 2020. +Prior to that, it was basically a direct result of COVID. +Prior to that, I was a live sound engineer. +So setting up for concerts and corporate events, all that. +All that went away pretty much overnight in March 2020. +So yeah, I started learning programming, landed on Flutter. +That's my main thing. +Fast forward a couple years, I had finished up a contract with a company called T-Vision Insights. +Luckily for me, most of their services are written in Python. +One of the lead engineers there is a fan of both your podcasts, Michael, and gave me a tip that you're looking for a Flutter developer, which brings me here. +So I was involved in building the Tuck Python mobile app. +Yeah, now I'm on the Flutter team for an insurance company. +I did not write any Python until I had finished the mobile app. +I had sat through a lot of those courses throughout the development process, kind of landed on FastAPI for my backend services. +Really like it, was really impressed with how easy it was to get started and all that. +So yeah, that's where I'm at. +Yeah, awesome. +I'm really looking forward to talking to you, sort of the interplay of how we work together to build that mobile app. +And it's just so well-received. +It's such a solid app. +The only people who don't seem to receive it well is the app store people. +We'll save that for later. +If anyone wants to see me cry, there's a good chance at the end of this episode, I will just be in tears. +But we'll save that for later. +But I do want to ask, I mean, I think that's a really interesting background and people, a lot of people listen to this show and they feel like, they hear from guests who are like, yeah, I got my computer science degree. +I was like an accelerated student. +So I got it when I was 16. +And then, you know, like, they're like, well, I admire this person and they're doing awesome stuff. +But I didn't get my computer science degree at 16. +Like I was still in high school and I didn't start programming until I was 30 or, you know, and is there still a place for me? +And I think you've super excelled at that in honestly, with Flutter and Dart, which is not one of the simpler beginner, more beginner friendly language. +Like it's a pretty seriously structured language. +So maybe just, you know, give people your thoughts of your three year journey, I guess at this point. +First of all, thank you for saying that. +And yeah, I listened to your podcast. +You have some very impressive people on your show. +My first line of code till 36, right? +It was weird, a weird time 2020, obviously. +But to be fair, I did have a lot of time on my hands, like many of us did, you know, very abruptly. +So I kind of went all in, you know, that's just I got hooked fast. +And I had a lot of fun. +You know, it's kind of thing not everybody gravitates towards programming, they try it, they don't like it. +It's not for everybody, to be fair. +But all it is, is consistency and putting in the time. +If you don't have a degree, it's fine. +I even chose Flutter is not like if you're looking for a job, there are better technologies to choose if you're just trying to, you know, land a job, right? +I'm obviously way more ubiquitous in terms of finding a job, even on the front end, learning something like React, way more jobs. +But all that to say, it's still possible. +I didn't go to college for audio or programming. +But again, putting the time consistency, the job hunting process was not particularly fun. +It's pretty grueling. +But sooner or later, you just keep at it and you can get there. +So it's never too late to get started. +Awesome. +Yeah, I know you're doing good stuff these days. +That's great. +Thanks, Jeffrey and Joshua. +First of all, my name is Joshua. +His name is Jeffrey and Jeffrey and Julian. +We are from Colombia and he is a Spanish speaker. +So he doesn't know to speak English. +So I'm here to help. +Sure, sure. +Yes. +Oh, wonderful. +Well, welcome to both of you. +Okay, thank you. +Tell us a bit about yourselves. +Okay. +Julian, who is the main programmer or designer of his app, started like 15 years ago when he was playing some video games and he wanted to know how did they work. +So he started to look for it and started programming, you know, like self-taught people. +And that's it. +He started a career, a university career, and he's a very talented designer. +And also, you know, as a friend. +So he is a very close friend of mine. +And I can say you, I can say, I can tell you that I think maybe the right place to start our conversation. +Now everybody knows all of you is just a quick introduction to your apps. +And then we'll dive into maybe the first thing is everyone here has chosen a different framework, a different language. +I think that is in and of itself is pretty interesting that we're all here as fans of Python or with some kind of Python roots or interests. +And yet that is not, you know, it's not necessarily what we chose to write the apps in. +So I think that's going to be a real interesting conversation, but let's jump over and maybe start with this one. +So PinPlanet, I forgot which app goes with which person or group. +So we'll have to just jump in. +Who's PinPlanet? +I'm the creator of PinPlanet. +The idea started from, actually started from my girlfriend in December 2020 or October 2021. +I'm sorry. +She's a big traveler. +She's always wanted a way to kind of curate all of her travels, kind of keep them in one place. +And the thing that she told me at the time that really sold me was, I want to be able to show all my friends, like my place I've traveled to with pins on a 3D globe that spins, which is actually the second screenshot on the screen for, I guess, for people who are watching the live stream. +And then from there I was just, I was sold. +So I started, I started building this. +I will say that the original version of this was actually a web app, a progressive web app, and I was a huge proponent of it at the time. +But for anybody who's used PWAs, they know the experience on both Apple and Google. +Somehow both of them are just not that great. +You just cannot seem to win. +And we've trained users for what, like a decade to look for apps in the app store. +So that's also kind of a pain. +Yeah. +And you can't really list your progressive web app in the app store. +Right. +I think actually progressive web apps might be passable because there's a lot of bad frameworks out there that it would, they would compete with, but the discoverability is, is the killer, right? +It's quite tough. +So I ended up building, building this app about three or four months into it. +I brought my friend Andrew as well. +I'm actually sitting in his office right now. +He does quite a lot of Python as well. +More Python than I do. +He's the person that I kind of look up to when it comes to, to writing. +I can't forget the, I forgot the word for it now. +Not Pythonista, but it'll, it'll come to me. +Oh, writing more Pythonic code. +Yes. +And then from there, some, I think somewhere around the summer I decided, okay, you know what, this is the app is gaining a lot of traction, but people want a native experience. +So I kind of took the plunge, learned Swift and SwiftUI, which is a really awesome framework released by Apple. +And then we, we released the, the native version, which focuses on like, you can pin all your places. +You can add your spots, your reviews, your photos. +You can add travel buddies. +You guys can collaborate on trips together. +There's a really cool explore page where you can see kind of the best pins that other people have put up. +And we're just, you know, adding more features over time. +It's gaining traction. +It's a lot of fun. +It's really exciting. +Awesome. +Congratulations. +Thank you. +I see that it's a, well, first of all, it has five star, a five star review out of a bunch of ratings, which is pretty ridiculous. +That's awesome. +Also that it's free though. +Is it just free free? +Is there in-app purchases or what's the story? +It's completely free. +Just looking for growth and traction and you'll figure it out. +Yeah. +Oh, actually, since you're on the screenshots, Michael, I do want to say thank you for the tip, the screenshots pro website or web app. +Oh yeah. +That helped out quite a bit. +Yeah. +Really appreciate it. +Well, it's a pay it forward because Lauren suggested that to me from his app. +So it goes. +And on, on it goes. +Yeah. +So I think it was screenshot, I believe was screenshots pro. +Let me screenshots.pro is the app that I use. +Yeah. +You can come up with really fantastic looking screenshots. +I'm super happy with how that came out. +All right. +My club. +That's mine. +Yeah. +And my club is, like I said, previously, this is a membership management program. +We've previously been working with progressive web apps as well, but we had a Ruby on Rails developer who did those, and we are a Django backend in where most of the information was. +So we had a lot of different servers connecting to our backend and doing all of these strange things instead of just having one. +And exactly like Harut talked about, you need to have your app on the app store because there's a lot of people asking, where is your app? +Even though we did have like progressive web apps and you were able to see everything. +But what we have now is we've developed so that all of our users can handle the members with their invoices, their memberships in various groups and so on to handle if they're going to certain trainings or going to certain meetups and just answer to that in the app. +And we also have like push notifications and all of that as well. +Yeah. +I think one of the main reasons people might want to create an app as a business is for the notifications because that's an interesting way to reach out for sure. +It really has helped a lot as well. +I mean, just being on the app store, regardless if you're on Android or on iOS, just being there actually means that things start to happen with the usage and so on. +We have about, I think in total about 8,000 installs on these apps, which from going from nothing about half a year ago is rather good for us. +We're a rather small company, so it's... +Yeah, that's really good. +We are, everything's in Swedish. +We are a Swedish company and we don't have anything translated into English yet. +The screenshots are awesome. +And I think it's, it makes perfect sense, right? +Like not everything needs to be in English, even though it does seem like it some of the times I suppose. +All right. +V3D. +Let's talk about that guys. +Jeffrey is going to talk in Spanish and I'm going to translate. +Yeah. +Thanks. +V3D es una aplicación que inicié en el año 2020. +V3D es una aplicación que comenzó a diseñar en la pandemia de 2020. +Inicialmente era para mostrar gráficos 3D, ya que como hobby soy modelador 3D en el software de Blender. +It was initially just to show graphics on 3D because he is a developer for this. +Poco a poco vinieron más ideas y quise seguir la programación, llevándolas al punto en el que está hoy, donde soporta texturizado, creación de materiales, también soporta mando de control Xbox y tiene un módulo de realidad virtual. +And by step I came up with some ideas to implement to the app, like featuring and modeling. +We can use an Xbox controller. +Yes, it has a lot of features. +That's pretty cool to use the Xbox controller with the mobile devices. +Yeah. +Actualmente estoy trabajando en una actualización que involucra texturizado con imágenes en movimiento. +Sigo trabajando en ella y próximamente será lanzada. +I'm currently working on animated textures, features. +Yes, I'm working on it and very soon it's going to be ready. +Yeah, this looks like an amazing app. +This is built in Kivy, yeah? +Exactly. +Awesome. +This portion of Talk Python to Me is brought to you by Sentry. +Is your Python application fast or does it sometimes suffer from slowdowns and unexpected latency? +Does this usually only happen in production? +It's really tough to track down the problems at that point, isn't it? +If you've looked at APM, application performance monitoring products before, they may have felt out of place for software teams. +Many of them are more focused on legacy problems made for ops and infrastructure teams to keep their infrastructure and services up and running. +Sentry has just launched their new APM service. +In Sentry's approach to application monitoring is focused on being actionable, affordable and actually built for developers, whether it's a slow running query or latent payment endpoint that's at risk of timing out and causing sales to tank. +Sentry removes the complexity and does the analysis for you, surfacing the most critical performance issues so you can address them immediately. +Most legacy APM tools focus on an ingest everything approach, resulting in high storage costs, noisy environments and an enormous amount of telemetry data most developers will never need to analyze. +Sentry has taken a different approach, building the most affordable APM solution in the market. +They remove the noise and extract the maximum value out of your performance data while passing the savings directly on to you, especially for Talk Python listeners who use the code Talk Python. +So get started at Talk Python.fm slash Sentry and be sure to use their code Talk Python, all lowercase, so you let them know that you heard about them from us. +My thanks to Sentry for keeping this podcast going strong. +Now, Lauren, I was going to pull up, I will pull up the Talk Python app that we built together, the mobile app that you built and the backend stuff that I built. +You also have an app that you recently built just straight in Flutter. +Maybe just pull that up real quick as well. +I have it installed and I like it, but I don't remember exactly what to search for. +Oh, thanks. +Yeah. +It's called Epic Skies. +It's just a weather app. +Epic Skies. +That's right. +Yeah. +That's a beautiful app as well. +So you've got a couple apps out in the app store now, right? +Well, yeah, this and as far as apps that I built myself. +Yeah, this one, which was released maybe a month ago and your app, Talk Python app. +So this one is straight Flutter, not too many backends, right? +Probably consumes like some public weather API data somewhere. +Yeah. +But that being said, having taken your async Mongo Python course, FastAPI, as I go to implement push notifications, the backend logic will be in Python and determining who gets what local weather alerts, all that will be, the app is already deployed. +I just don't have the features. +I don't have all that functionality built yet. +But yeah, backend for this handling the notifications will be Python. +That's really cool. +Yeah. +You had a couple of releases this year so far. +And so then also I'll let you talk about it since you built most of the front end stuff. +Yeah. +The Talk Python app. +Sure. +Yeah. +I imagine most of the listeners are also students. +So, you know, maybe a lot of people listening, I've used it, but basically it's a mobile app that allows students of Talk Python Training to consume their video courses, right. +Similar to Udemy or something like that. +Yeah. +So we have our very fast backend written in Python, pulls in the user info, what courses they have access to. +We track what courses, what lectures the users have watched so that it's in sync. +If they go to the web app, all that is, you know, stays nicely in sync. +Yeah. +Written completely in Flutter. +There were no, you know, it relies very heavily on video player, obviously, but all these libraries that we rely very heavily on are just available as Dart libraries. +So I didn't have to write, you know, not a single line of native code as far as that goes. +Even, you know, when Apple made us do in-app purchases or, for example, showing the progress in the notification bar, there's libraries for all of that. +I'm grateful you chose Flutter as this, cause I got to be involved, but I also think it was an excellent choice that you could have made. +I think it was an excellent choice as well. +And let's go and talk about that. +Cause I can see some good questions out in the audience and I know this, and I also opened the show like this, right? +Like there are many ways in which you can build these apps and Lauren was just touching on it. +How close do you need to be to native and use the exact API delivered by Apple or Google? +Or how much do you care about cross platform? +Because on this one, for example, you can get it on Google play, you can get it on the app store. +The only real changes is there's different rules. +And so for example, you can't talk about prices or you've got to do in-app purchases on Apple or, you know, there's, there's little variations that they force from their policies onto you, but otherwise it's just the same code, same code for iPad and tablet as it is for mobile, as opposed to Haru, who has the Swift project, which is awesome, but you're really tied to Apple, but you also get to take advantage of the closest changes. +The build published stuff doesn't keep running into weird issues when there's some mismatch. +I mean, you want to strike fear into your heart, just mentioned new release of Xcode and then that literally just happened this week. +I was building something on Xcode 15 beta eight, and it was a huge pain to install. +Didn't build for no reason. +It's completely unclear, no documentation whatsoever. +It's not great. +So let's talk about choosing the framework and I'll just go through the, I got a bunch of tabs pulled up so we can talk about them. +Andreas, you chose Ionic. +Tell people what Ionic is and then why you chose it. +Ionic is a way to create applications that can be delivered both to like normal website, to Android and to iOS. +So we get all of the functionality and only need to write the code once. +What it really is in our case, we chose to write everything in Angular. +It's a web view running on the mobile phones. +So you actually get just a simple server running on the mobile phone and also on iPads, of course. +You can get connectors to the phone for doing things like push notifications. +You can connect to the photo and photo library and also to the web camera and so on. +So you can get all of that functionality into your app as well. +The only difference being is that you only have to implement it once and it works on all of these three platforms. +You can even get web functionality or sorry, camera functionality on the web application as well. +That is running in an Angular, in a normal Angular platform. +So yeah, that's and the reason we chose Ionic was that we're a very small team, only three developers. +I and one of the me and the other one of the other developers had some experience of Angular. +I've released other applications in Angular and we decided that that would be the way to go. +So yeah, that's really why we chose Ionic. +And what it does have it has like style sheets for each type of application. +So you have one style sheet for the web, you have one style sheet for iOS and you have one style sheet for Android. +So each button gets styled. +So it looks like an Android button or an iOS button and so on in the system as well. +Are you all using the Ionic framework for your web app as well? +Yes. +Okay. +Now that's interesting. +That's a different level of integration there that or a reuse. +That's pretty interesting. +The thing is for us, just this specific application, because what we did have previously, we had a progressive web app that was written on Ruby on Rails and the Ruby on Rails developer decided to go elsewhere, which meant that me and another Python guy, Ruby on Rails application that we had to take care of. +I have read some Ruby on Rails. +I've also written some Ruby on Rails previously, but it's like, is that 15 years ago, I think? +So that's not something that we wanted to keep up with. +And what we do need to do, we need the deep linking, which means that if you get an URL in an email, because we send out notifications for invoices, for example, you get an URL in your email, you click on that, you either go to the web application, if you don't have the app installed on your mobile phone, but you can get into the application directly if you have it installed on your phone. +So that type of integration is very important for us as well. +It's worth pointing out Ionic is cross platform, not just web to mobile as I'm learning, but also so you can publish in the Google Play Store, you can publish in the Apple App Store. +We publish in all app stores and then we also have it on the web on its own URL as well. +It needs certain documents in the web application to actually make sure that deep linking works in the mobile space as well, so that we can get to the right mobile phone and application and so on as well. +Did you look at other frameworks? +Were you thinking of maybe going true native and writing in Swift or other things? +The main thing is, like I said, we're a small team and we needed to do something that was rather simple, things that we knew something about. +The other main developer, he is Python Django developer. +I worked with him also on that online tailored suit company that we had previously. +And we've been working with Django for a very long time. +And just changing to like writing something in Swift or writing, then needing to also do it in Java for an Android application would be too much hassle for us. +And therefore we chose something that we sort of thought that we would be more or less comfortable with. +It's still just HTML, that's CSS and it's some TypeScript, which is like a better version of JavaScript. +What is your framework? +I mean, you kind of mentioned this a little bit, but why did you choose this? +And like, you know, did you look at others? +And yeah, I think this is actually the biggest choice people have to make when they're building an app. +It's like, what do I even, which direction do I even go? +Even if I'm sure I'm doing Python in the backend, then still that doesn't solve this problem necessarily. +I think the story will make a little bit more sense. +I mentioned earlier that the initial version was a PWA. +That's mostly because my background is web development. +I write a lot of Vue as well. +However, the PWA was difficult for discoverability. +And then I had to make the really tough choice of, okay, what's kind of the next thing to do? +I really did look kind of all around the landscape. +But the one thing that always kind of scared me was that if, or I guess the thing that really pushed me into going native was like, if I wanted really native functionality, which, you know, having like been billing apps for quite a few years now, sometimes the things that you want to do or the things that people ask for veers off into like the, all right, this is actually pretty custom. +There's no library for this. +Like we got to, we got to just like roll up our sleeves and do it. +Yeah. +I knew that if I had gone cross platform that all my experience, let's say for example, would have been in Flutter or React Native, which are like the two really, really popular options. +And if I now needed to write something custom, the one year of Swift experience that I have now I wouldn't have had. +So it's like, not only am I using a really difficult and probably not well-documented API, it's like, on top of that, I also don't even understand the language. +So I think for me, that was kind of the big, the big thing. +And one of the reasons was I always thought maybe like, we haven't done this yet, but maybe we could like amp up the camera or do something with video players, or maybe like be a little bit more custom with, with the way that we pull, pull our images in and take more advantage of caching and stuff. +So that was where kind of, that was the reason why I ended up, I ended up going with Swift. +I think it was, it was a pretty cool choice. +Yeah. +And it sounds like SwiftUI makes it a little bit easier than, you know, a bunch of storyboards and all that. +Let me say a little bit about, about SwiftUI. +It has been quite helpful. +I don't know if it's because I've come from the web. +I've heard online that the people at Apple or the, the people who develop SwiftUI were influenced by React. +I could see that a little bit having, having used a little bit of React, but it is, it is quite a bit more, it's a lot more expressive. +It's a lot easier to write. +But let me also say that you're not going to get away with only writing SwiftUI. +Like you could do maybe 80 to 90%, but there's that additional 10% you have to dip down to UIKit and, and kind of put some extra stuff together. +Still a little rough around the edges. +That's kind of the case. +I'm always thinking of new apps and new ideas and that I have no time to work on them. +But the one that I have no time to work on right now that I'm thinking of is something to do with the Apple watch. +And there's no Flutter for Apple watch right now. +So I'm like, well, maybe I'm going to have to learn Swift after all. +We'll see. +We'll see if I, first I got to find time and then we'll come to that. +We'll solve that problem later. +Awesome. +All right, Geoffrey, Kivi, why'd you choose Kivi? +Mi framework favorito en ese caso es Kivi. +Kivi lo conocí en el año 2011. +Okay. +My favorite framework is Kivi. +I knew it in 2011 and I use it because it's very stable and steady. +And because it has a very active community and it allows and develop applications for many platforms and with one use, just one code. +So that's why he chose Kivi. +It seems like Kivi is a good fit for the type of application that Geoffrey built. +So it was a good match. +Yes. +Si, totalmente encaja con lo que he buscado. +De hecho para mi es gratificante que lo conjundan con un juego porque siempre he querido que tenga ese nivel de realismo que ofrecen los videojuegos como los frameworks de Unity 3D. +Okay, it's grateful for me that Kivi really match with I look for because I found it nice that people can feel it like a game because games are realistic in some ways to control what do you want, how do you want to and custom your car, for example. +And this application gives you lots of features to implement to what you want. +Okay, Lauren. +I don't remember if I mentioned it. +I actually started briefly with Kotlin and native Android programming in 2020. +But once I tried Flutter, I just found the developer experience to be exponentially better, right? +And with the added benefit of being cross platform. +So Kotlin as a language, I think is great. +It's not that's not the issue, but just the overall process of building an app and native Android versus Flutter. +I imagine I would probably like Swift better. +If I tried that I haven't really had a need to dive too deep into it so far. +But I think Haru made some interesting points in terms of you need something really custom. +Yeah, he would have had he chose a cross platform framework, he then would have had to learn Swift and then do all that. +But I do want to make the point that you do still have access to any native API's that you need, right? +Even if there's not a library for it. +And Flutter, it's something called a method channel. +And then you can just just a message you send back to the native platform, whatever you got to do on the native side, you can do that being said, in terms of what really struck me in Flutter was building UI was so much nicer than than doing it Android with a hot reload, you know, you make a change, hit save immediately updates the widget. +So in Flutter, all your UI components are called widgets. +If you ever look at some Flutter UI code, it's I didn't really like it at first, like it looks a little bit weird if you're not used to it. +But as soon as you wrap your head around the widget tree and how it all works, and get used to using snippets and your preferred IDE, you just say wrap this widget and another widget that saves a bunch of time. +It's really not fun if you're doing that stuff manually without without snippets. +But yeah, all that being said, I was just so much more productive building UIs and again, having the added benefit of being cross platform. +So for me, that's I mean, it's unlikely I'll ever have a need to go native or choose a different framework for that matter, because Flutter suits all my needs. +So there's some code that somebody has never seen that that might look might look a little bit weird, right? +It looks normal to me. +What's interesting about it is like, it's really you build these, these like hierarchies of code, but they exactly match the UI hierarchy, like this panel contains these three things, and they're kind of indented in code. +But boy, do you need a smart plugin in your IDE or your editor to like manage the back end of those curly closing things? +Yeah, it was especially starting out, you're missing one, one parentheses or one curly bracket and your page is full of red squiggly lines. +And if you don't know how to sort that out, it's not once you get used to it, it's fine. +But especially in the early days, sorting out those closing brackets are pretty annoying. +But yeah, all that being said, I found myself way more productive with Flutter, really easy to build nice UIs, the performance is always getting better animations, all that stuff is really nice. +And you still have access to the full native API's if you if you need to use it. +Doc Python to me is partially supported by our training courses. +Python's async and parallel programming support is highly underrated. +Have you shied away from the amazing new async and await keywords because you've heard it's way too complicated or that it's just not worth the effort? +With the right workloads, 100 times speed up is totally possible with minor changes to your code. +But you do need to understand the internals. +And that's why our course async techniques and examples in Python show you how to write async code successfully, as well as how it works. +Get started with async and await today with our course at talkpython.fm slash async. +One of my concerns with all of these frameworks, these these non, non Kotlin, non Swift frameworks is like, well, how supported are they? +Like, am I excited about something that 10 people are using? +You know what I mean? +Right so I can pull up Ionic and Ionic's got 50,000 stars. +And then, you know, one of the other ones I definitely considered and chose is Flutter and it's got 157,000 stars. +By way of comparison, Flask has around, I think, 50,000, 60,000. +CPython itself has 50,000. +Right? +So like, oh, okay, well that seems like a good enough number that people are using it. +These are all really interesting choices. +And I think just making this choice, it's where you got to start, but it's a challenge. +You got to think about what you're building and whatnot. +Right? +I'd like to also add to that. +I mean, one of the main things that like you said, one of the things that we had a bit of a worry about was, will like, for example, iOS, the app store, will they accept something that's coming from Ionic framework? +Because you are literally just running a web application in a web view in your app is really a small component. +And we even had in the previous versions, we had the ability to update the underlying HTML code and the application code from a server that is on the internet. +So it was really a really strange thing, but Apple has never had any issues with our application. +Yeah, that's fantastic. +Congratulations also. +I know, I don't think it's the Ionic framework. +I think there's always a weird issue. +There's just a weird review processes there. +But I think Ionic seems to be pretty solid and that people have apps out in the app store. +Flutter, absolutely. +Like you can see a bunch of things like the BMW app and stuff. +Kivy also, I think that's really good. +I think there's, I think one of the rules of thumbs that people should really have is if you're going to use a framework, just look to see that there are apps in the app store based on that framework. +Right? +Not proof of concept, not theoretical things, but like here are apps in the app store based on this, this app. +And not necessarily because it's only a good thing if the app is in the app store, but there's all these little edges, right? +Like I said, in-app purchasing, for example, right? +That's something you could, you could build a cool mobile framework and never add that because you haven't gotten around to it. +But if, even if you don't want it, if the people at the app store require you to have it, your app doesn't go in the app store until you put it in, regardless of whether you even want to. +Right? +So I remember Lauren and I spent, how long did we spend? +Like a week and a half full-time adding in-app purchasing and we didn't even want it. +Right? +And they're like, Nope, Nope. +You have to have it rejected, rejected, rejected. +Nope. +Here's the way you can, what is it? +You can like protest this rejection to a higher authority, protest it, reject it. +You know, it's like, it wasn't good. +It wasn't good. +So you got to think about those kinds of things, like when you choose a framework, right? +And good job on that in the end, Lauren. +We're not done as, as we'll see. +But yeah, nonetheless, we're, we're pretty much there. +Again, I had a blast working on this. +It's really, really nice working with you directly. +Really grateful to have been a part of that. +There's a question in the audience. +I kind of want to get, I guess this is for you, Lauren. +This is Alan asks, looking into Flutter, I want to know the best way to integrate with AWS, for example, is Cognito an option for a signup flow? +So I don't know what Cognito is. +I guess it's an authentication platform from AWS, I'm guessing. +But yeah, I mean, just maybe talk about that real quick. +If, if you've got any experience, I don't personally have a lot of AWS experience and I know they have like a million separate services. +So I do know that there is an AWS library for Flutter. +Again, I don't know what exactly that entails in terms of which one of their services it provides a Dart wrapper for, but either way, it's certainly possible to integrate with any, any other services. +Even if you have to, you know, do it from scratch, it's obviously possible. +It's all just APIs at the end of the day. +So I would say that like every one of these frameworks we just talked about, to my knowledge has something like pip and PyPI, right? +PyPI, we're all used to pip install our projects. +Like that's just the lifeblood of how Python works. +But for example, Flutter has pub.dev and you can, you know, pub, pub get stuff and it's in the equivalent type thing, right? +Compared to like React Native, which is a little bit older, you know, you, you see a lot about how they're, they have a more robust package system, but in my experience, I've yet to run across a scenario where I did not find the library that I needed. +Like weather app uses location. +It's plenty of location libraries, you know? +Yeah. +And Ionic has NPM, obviously, right? +Yeah. +And they also have their own package. +They have the Cordova packages or Cordova. +Okay. +Yeah. +Which are, are what they use for connecting to the telephone itself. +So everything that you need to do with that, you, you go through the Cordova store. +There are a lot of various packages on the Ionic website themselves. +So, yep. +And to varying degrees of success, we have CocoaPods for you Haru, right? +Now it's actually Swift package manager. +So it's, it's a little bit better. +I think it goes by SPM or I don't know. +Yeah. +A little bit easier to install less configuration hassle overall. +Because like I said, the CocoaPods are kind of a little bit janky, right? +They're based on like Ruby gems and then they compile it. +A lot of Rube Goldberg and stuff happening there. +And then for packages in Kivy, you just use pip. +Yes. +Utilize la libreria Pginus. +I use the Pginus library. +Y tambien utilice para monetizar la aplicacion en la play store. +Con publicidad utilice Kipmo. +And for play store, I use for purchasing Kipmo. +Cool. +So there's third party packages that you can do in-app purchases with. +That's great. +Utilizar graficos 3D, calculos rapidos NumPy. +And for 3D calculation, I use NumPy. +That's really cool. +That's one of the most popular ones for general science, but I guess it's, it's neat that it applies right there on mobile apps. +There definitely are some advantages to having Kivy. +This is a question for you, Andreas, I guess, but we can maybe all talk about this real quick. +What are the limitations of Ionic regarding to being connected to the internet versus I being offline? +I'm guessing that that question has comes from the fact that it sounds like a web app framework. +It is a web app framework, but it is self-contained. +So when you install the app, you get all of the JavaScript, all of the HTML, all of the CSS that's required to make your app look like it should look on the mobile phone. +It's more of how you design your application yourself. +If you want, for example, our application doesn't work without internet access because we have all of the information on the backend and there's really nothing that we can do without internet access. +But you could of course design it that you download stuff and you use it with, there's a lot of plugins to get like in, what is it that Apple likes? +They like to use the default database that we get on all of our Python. +SQLite maybe? +Yeah. +SQLite. +Thank you. +They use a lot of SQLite on the apps and you can get SQLite connectors and use that locally on the mobile phone itself. +So you can store everything if you want to and do a complete offline experience as well. +And you use block, but did you use block for offline capabilities? +For anyone not familiar, block is one of the many available state management libraries for Flutter. +That's the one I prefer generally. +It's what I use at work as well. +They also have a library called hydrated block, which makes it really, really easy. +It just persists any state change within your application without having to write manually write your code. +So basically it's a to JSON and from JSON situation. +You have like a state class and as long as your state can be serialized in the JSON, it just does all that in the background, which is really nice. +So there's like that kind of state persistence between app sessions, but then there's also actually downloading the files like for when the, you know, offline was very important to you, right? +As far as the functionality. +So yeah, yeah. +That's one of the reasons the app existed. +One of the two or three, right? +Sure. +Yeah. +So the thing is that wasn't really hydrated block was not actually it saved the file path, but the actual bytes, you know, streaming of the bytes and saving the file itself. +That was just basically just the HTTP library, writing that directly to the, to the device. +And then hydrated block saves the file path. +The next time the user needs to watch that offline. +What's your offline story? +We don't really have an offline story. +I think it's kind of similar to like other platforms. +So something like Instagram, if you don't have an internet connection, you don't really see photos and it's kind of similar for us right now. +Keep it simple. +Right. +And Jeffrin, does your app use online features or is it just offline? +With the Lisa and the third thing is offline. +In v3d we use offline images. +Okay. +Yeah. +So no real internet. +Excellent. +All right. +Let's do talk a little bit about some Python, about what's going on here. +Since you know, we're just a little ways into the show, I guess we could talk some Python at this point. +Yeah. +It's been a great conversation. +So let's go kind of top to bottom. +And Andreas, it sounds like yours is very much an online application and has backend services. +You already talked about the Ruby on Rails app and getting away from that. +Tell us your story. +Give us a little sense of like, what is your backend architecture and just, you know, what do you got going on with Python? +Where some of the technologies and things there. +Our backend architecture is actually just changed. +We're really proud of it now because we've moved from our own installed servers that have been installed with via very long scripts that you've written down exactly what you need to do and everything and running our own MySQL servers and so on. +And we've moved everything into AWS and we're currently running in a Kubernetes setup. +So we are really proud of how that's running currently. +But we are a Django shop. +We've been running Django. +The first version was written in 2013 and we've just iterated since then. +And we're now currently on the latest versions of all of the dependencies that we have in PyPI as of last week, because we did a major upgrade. +We need to update our Redis connections for certain reasons. +But yeah, we've been running with Django and just doing a regular Django application really. +And that's the main part of the admin for our customers. +They use the Django app to do their administration for the memberships, sending out invoices, selecting activities that they need their members to join and so on. +All of that is done more or less currently in the Django app. +And then we have the mobile application is for the members to go in and check which activities are we going to go to, which invoices do we need to pay, connect to other members in the teams or in the clubs that they're members of. +Primarily, you've got this Django app. +And I would imagine that people can log into the Django app as well. +Yeah, and previously, we had like a lot of Ruby on Rails servers that did certain parts of the application. +For example, we have a lot of different payment solutions for the invoices you can pay via Swish, which is a comparison with I think it's called Venmo or Cash App in the States. +What you do is you send money via your mobile phone number in Sweden. +And so we have a backend for handling that that used to be written in Ruby. +We've moved that into Django as well. +We also have credit card payments, which was in Ruby on Rails moved into Django. +And so we're trying to consolidate a lot in the backend and try to move everything back into the Django application and just have the mobile apps out for the members to use to talk to the backend. +That's excellent. +Yeah, there's a bit of a trend away from all that stuff, right? +There's a great article, give me back my monolith. +Yeah, exactly. +Instead of all the microservices and different technologies, just give me one thing, just let me put it into the app. +But it sounds like kind of like you embrace that with Django. +We've got a lot of experience with Django. +Me and the other main developer, we've been working with Django for, I've been working with about 10 years now. +He has been working about 15. +And it's our preferred go to solution for everything in the backend. +You must have APIs and stuff, right? +Do you use like, what do you use for the API side of things? +Is it still just straight Django? +Or do you have some REST frameworks? +We use Django with Django REST framework. +So it's really a simple solution for that as well. +And we really enjoy working with that. +We have another guy who's come in now who is a developer I've worked with a lot from the Ukraine. +And he's been working with us now from Poland instead, just doing development with Django REST framework. +And he's really good at those things. +Yeah, it seems like a nice one. +Have you considered Django Ninja? +I've just read about it. +I haven't actually done anything with it. +I read about it the other week. +If you like fast API, but you also like Django, then maybe Django Ninja is what you're looking for. +Yeah, exactly. +But that being said, I mean, the new releases with Django with version five with all of the now async views, everything that we're getting with that, I think Django is one of these things that you know that your code will always work. +But you also get that you're starting to get all of these new features as well to get the systems working as well, which is really amazing. +I think they're doing a great job with the Django community now. +Excellent. +So similar to Andres, we also use Django. +So I'm just going to be basically talking building on top of a lot of the stuff that he said, big fan of the framework. +I've been using it for years. +I'm trying to think we mostly use Django REST framework. +A lot of the stuff, a lot of the views are kind of custom built. +The ORM is very helpful. +Migrations are really helpful. +We do write some raw SQL for some of the more trickier parts of the app, like the explore page where the queries need to be more performant. +We use HTTPX to send notifications to APNS, which is Apple's push notification server. +We are big fans of type hints. +I have some of the stuff written down here, just to kind of point out. +Post GIS for some of our geospatial calculations. +And most of it runs on, I don't know why I'm saying most of it, all of it runs on AWS on pretty bare bones EC2 servers. +I kind of like the simpler monolith approach where you kind of just like you have a bare bone server, you set up everything, and then you can use something like Ansible to help you bring it up and bring it back down. +But it's just a lot easier to keep in mind. +And deploy and version and all those things. +Hey, while you're on, I did got a question on the audience I'll go ahead and ask. +In your pin app, is it difficult to write code so that changes are saved? +Lauren talked about the block thing, you kind of just make the changes to the state and they just stay. +But what's the story for yours? +I don't know if I fully understand the question. +But I think there are two ways, two things that I could say to answer it. +One is we use user defaults and Swift to take advantage of like saving stuff directly on the device. +But actually, the main thing that we do is we push a lot of logic to the server. +And this is like my favorite way to kind of get around the App Store nonsense. +Just I guess is the nice way to say it. +But the more and more logic that you push to the server means that whenever you need you find a bug or you have a feature or fix that you want to push out, you know, you can just change it on your server, redeploy and then boom, like it's available. +So sometimes like when people on the app complain about issues with like, oh, this is not displaying correctly and then I'm like, oh, just refresh the app. +And they're like, oh, that's crazy. +I can't believe that just worked. +And then other times certain things that I have to do on the device, I'm like, oh, well, here's a now I got to submit this to the App Store or wait a day for it to get approved, wait two weeks for everybody's phone to update. +Yeah, absolutely. +You can do it on the server. +It's just have so much more control. +And I mean, Lauren, that's how you and I worked a lot, right? +A lot of the things were like, well, if we want something to be visible in this collection or not visible in that collection, or, you know, like for example, if you want, say the free courses to appear in a different section, but not in the main one, you could just not return them in the API rather than changing the app potentially. +Right, exactly. +Yeah, I was actually, I recall being impressed by how easy it was for you to make these changes, you know, as these requirements kind of popped up throughout the process, you'd be back in 10 minutes, okay, it's ready, you know, and it was, I guess, Mongo is maybe partially a factor in that and how flexible it is. +I made zero database schema changes explicitly on this. +I just wrote code and it just started doing what it needed the whole thing through the app. +Yeah, the whole app dev process. +Yeah, it was fast. +I remember telling you when I started my job, it was noticeably not as fast as your backend. +And that's a big company with lots of money, that should be super fast. +Well, yeah, to be fair, I think it's reasonable to assume an insurance company has a larger and more complex backend. +I think the complexity is also crazy, right? +Like probably you've got to, we got to apply a bunch of rules to this question, not just return the data. +Yeah, sure. +Nonetheless, I'll say it again, your backend is crazy fast. +And you kind of got me hooked on the fast API Mongo situation. +Yeah, really impressed with all that. +I know you're not using fast API for your backend for that. +I would if I got a choice actually, but I mean, no, no, no, no, I'll go ahead and say a little bit about what we're doing on our backend, just to kind of be fair to round it out. +So the talk Python web app is written in pyramid. +I love the pyramid framework. +It has a lot of cool features. +It's really, really fast. +It was one of the very first ones to support Python three, like intentionally, not like it might run, but we embraced Python three. +That was a huge reason that I chose it. +I also love the chameleon framework for writing templates. +It's, it's really nice. +It's been great. +I like the projects with ambitions, start small, but finish big and stay finished. +Stay finished. +So that's really good. +And then it talks to MongoDB using Beanie and that combination is just great. +And like Haru, it runs on digital ocean, not on, not on AWS, but it's just a couple of servers, right? +One for the front end web server, one for the database and it's golden. +It goes great. +So yeah, that's, that's pretty much what we got going on there as well. +Yeah, I probably would choose fast API if I were to start over, but rewriting the web app, it's like 20 or 30,000 lines of Python. +And you know, it's, it would be cool if it was fast API, but I'm not sure that the benefits of making all those changes are really there. +It's, it's like, I already liked the framework as it is. +I also came across light star recently. +I just interviewed those guys here on the show, but not yet published. +So in the past, but also in the future, in a sense, and this is a really interesting thing also built on starlet that I'm really, really interested in and checking out more as well. +So there's a lot of, there's so many good backend frameworks to choose from, but you know, you got to pick one and go with it. +And then Jeffrin, I guess maybe it's worth pointing out that Kivy is while you don't have online capabilities for your app, Kivy itself is Python. +So yours is the only mobile app that is all Python. +Whereas everybody else's mobile app is something else. +Yeah. +So Kivy is a construal code Python and Kivy is building Python. +Con servicios en la nube construidos en flask. +With services on the cloud and flask. +Okay. +Flask. +Y también utilizo Dropbox en Python para el almacenamiento de archivos en la nube. +And I also use Dropbox to save data in the cloud with the same Python. +What does your app use the flask online service for? +Son utilizados para realizar la publicación de modelos 3D en la nube y que se puedan descargar desde cualquier otro lado del mundo. +We use flask to upload everything in the cloud and in order to be downloaded in any part of the world. +I said we would talk about the app store, but I also want to finish this on a positive note. +So I think I'm going to not try to make it the focus of app store horror stories. +Although I just don't get why they make it so hard to build apps for these frameworks, especially Google. +I mean, especially Apple, but also Google in their own special way. +So instead, let's just round this out with maybe like a piece of advice or your thoughts on going from having your app to get it in the app store. +Andreas, you want to go first? +From our point of view, the main thing that has been a bit of a hassle was if you want to publish an iOS app, you need to have a Macintosh. +So we need to go out and buy a Mac. +That was one of the things. +But that being said, what we have done a lot now is we try to automate these things. +We try to automate pushing the app into the various app stores from GitHub, which we use as a repository for it. +I've successfully done that now for the Google Play, but I can't get the iOS app to build, unfortunately. +Yeah, that does not sound practical. +But that's one of the things I think you should invest some time in, because if you do get it working, you can get a lot of these hassle things out of the way. +And building it on a Mac, it's four or five clicks that I need to do to build it. +But I need to make sure that I have the right versions everywhere, and I make sure that I have set the new version of the application in all places I need to update that in, both on Android and on iOS, various config files I need to update. +So really looking into automating all of that, I think would be a good thing to try and do as much as possible. +Yeah, that sounds fantastic, actually. +Even having one of the app stores automated is really nice. +Yeah. +Haru, you mentioned builds, Xcode, good times. +We're going to try to end on a positive note. +I don't want to rag on Xcode too much. +Let's not talk about it. +But you do need Xcode to build your app to get it in the app store. +That's for sure. +And if you're like right now, we're what, like four days? +September 12th is the launch of iPhone 15, and iOS 17 will come out a few days after that. +If you want to try to build something for iOS 17, you do need Xcode 15, which is still in beta, beta 8. +And that can sometimes be tricky to build for, not everything. +Because like the videos that Apple puts out in June for WWDC, the API has actually changed since then. +And it's again, not clear sometimes. +Yeah, we're a little spoiled in the Python world about how it, if something once worked, it generally continues to work. +This next question here from Alan, maybe perfectly lines up your part, Lauren. +I've not yet published a Flutter app. +So really want to learn how this works, any automation available? +Yeah, maybe just app store advice and chime in on that potentially. +Short answer is yes, there are a few different CI CD platforms available for that. +Like for example, GitHub actions, you can get that set up where you push a command from a terminal and it does all this stuff in the backend and pushes to the store. +I personally have not, that's on my to do list. +I have not actually done that yet. +There's also CircleCI, there's CodeMagic, there's a few different services available. +All that is probably a little time consuming to set up. +Let's say you don't want to get into that, which you should, it's definitely worth it in the long run. +But let's just say like, we're not for talk Python, we're doing it quote unquote manually, it's still not that bad. +Like basically in Xcode, you create the archive, upload it within Xcode, go on the web app, submit, right? +And it's more or less, you know, on the Android side, you're basically, as far as Flutter is concerned, it's when it comes to publish, you're publishing a native app, right? +So you just got to go through both those processes. +Android Studio, you build the APK and then upload, you know, and submit, right? +The other respective web app. +So even if you don't go there. +Yeah, it's worth pointing out, like, there's, I think, zero difference from the way you publish a native app and the way you publish these. +From what I do, like, I'll open up Android Studio and go through the steps or I'll go through Xcode and do this. +It just happens to be what's running or compiling behind the scenes had some Flutter component at some point, but it's, they're identical. +People should probably keep that in mind, right? +Yeah, it does compile to native code. +So as far as like the app stores are concerned, and as far, you know, it is a native app just with the UI being painted over top, right? +But so, but yeah, lots of options out there if you want to go that route. +Yeah. +And cross your fingers when, yeah. +Yeah. +Jeffrin is, is your app in the app store? +I know it's in the Google Play Store, right? +Google Play Store. +Yes. +Yeah. +We have online ship in app store, just only in Play Store. +And do you have advice for people getting their apps into Google Play? +Follow the guidelines of privacy. +Overall archives exploration. +Yes, that have given headaches for me since the updating of SDK. +Excellent. +Well, everyone, this has been really fun. +And I said, we're not going to share app store, horror stories, horror stories. +So we're going to finish this on a positive note. +I think in general, there's just so much possibility in mobile app development these days. +I don't know about you all, but when I first got first on iPhone or smartphone, like just my head was full of ideas of like, oh my gosh, you could do this, you could do that. +This is just such an open world for interesting things. +And it's really cool to hear how you're all building your different apps and putting them together. +So thank you for being here and it's been great. +Thanks for sharing your experiences. +Thanks a lot. +This was great. +Thanks, Michael. +This was a lot of fun. +Thank you. +This has been another episode of Talk Python to Me. +Thank you to our sponsors. +Be sure to check out what they're offering. +It really helps support the show. +Take some stress out of your life. +Get notified immediately about errors and performance issues in your web or mobile applications with Sentry. +Just visit talkpython.fm slash Sentry and get started for free. +And be sure to use the promo code Talk Python, all one word. +Want to level up your Python? +We have one of the largest catalogs of Python video courses over at Talk Python. +Our content ranges from true beginners to deeply advanced topics like memory and async. +And best of all, there's not a subscription in sight. +Check it out for yourself at training.talkpython.fm. +Be sure to subscribe to the show. +Open your favorite podcast app and search for Python. +You should be right at the top. +You can also find the iTunes feed at slash iTunes, the Google Play feed at slash Play, and the direct RSS feed at slash RSS on talkpython.fm. +We're live streaming most of our recordings these days. +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm slash YouTube. +This is your host, Michael Kennedy. +Thanks so much for listening. +I really appreciate it. +Now get out there and write some Python code. +you . diff --git a/flow_results/cleaned_transcripts/435-pypi-security_clean.txt b/flow_results/cleaned_transcripts/435-pypi-security_clean.txt new file mode 100644 index 0000000..c581467 --- /dev/null +++ b/flow_results/cleaned_transcripts/435-pypi-security_clean.txt @@ -0,0 +1,838 @@ +Do you worry about your developer data science supply chain safety? +All the packages for the Python ecosystem are much of what makes Python awesome. +But they are also a bit of an open door to your code and machine. +Luckily, the PSF is taking this seriously and hired Mike Fiedler as the full-time PyPI safety and security engineer, not to be confused with a security developer in residence tapped by Seth Michael Larsen. +Mike Fiedler is here to give us the state of PyPI security and their plans for the future. +This is Talk Python to Me, episode 435, recorded September 18th, 2023. +This is your host, Michael Kennedy. +Follow me on Mastodon, where I'm at mkennedy, and follow the podcast using at Talk Python, both on fosstodon.org. +Keep up with the show and listen to over seven years of past episodes at talkpython.fm. +We've started streaming most of our episodes live on YouTube. +Subscribe to our YouTube channel over at talkpython.fm slash YouTube to get notified about upcoming shows and be part of that episode. +This episode is sponsored by Sentry. +Don't let those errors go unnoticed. +Use Sentry. +Get started at talkpython.fm slash Sentry. +And it's also brought to you by us over at Talk Python Training. +Did you know that we have over 250 hours of Python courses? +Yeah, that's right. +Check them out at talkpython.fm slash courses. +Hey, Mike. +Hey, Michael. +It's awesome to have you here. +Thanks for having me. +I'm really excited to be here. +Yeah, I'm excited to have you. +It's interesting to talk about security. +I got to tell you, talking about security just makes me nervous. +Why is that? +Well, two reasons. +I feel like when you talk about security, you're kind of sticking your head up and people are like, let me see if I could whack that. +You know, not everybody, but a few people in the world, right? +But it is the internet. +So if you take a very, very small percentage and multiply it by billions, it becomes non-zero. +And then, you know, it's just one of those things. +It's like trying to prove the absence of something. +It's very hard to prove that you're not missing stuff, some step. +It's very hard to prove that you haven't, that there's not a, you know, you've got all the controls and there's not one control you forgot. +Right. +In that regard, probably more so. +It's pretty tricky. +The way I've often thought about security is it's a spectrum, right? +I used this quote a million years ago. +I don't know who said it first, but the most secure computer is powered off and buried in six feet of under concrete, right? +But it's useless, right? +It's very secure, but nothing in there is useful. +So if we take that as like a crazy extreme of secure and say the most insecure computer is, you know, powered on, has zero password control, connected to the internet and auto publishing IP data, so that way anyone can come and do whatever they want. +All right. +So that's the other end of the spectrum. +That's a really bad situation. +There's a fine balance that every software application system company has to kind of navigate to figure out where along those two crazy extremes, where do they fall and where are their kind of risk thresholds of tolerance are? +Like what would it cost me to add more security? +Well, I could, you know, lock down all of my users and not allow them in unless they come to the front door and show a picture ID, right? +Like, okay, if that's how we want to secure our building, that's one way to do it, but that'll slow down the ingress to our building. +So we issue our employees badge cards and we assume that they act in good faith and they don't kind of lose them and report if they lose them. +Oh, great. +So that's kind of a middle ground where you kind of delegate some of the security to the individuals and just kind of, you have to figure out where your security is and what you're willing to do and sacrifice in order to get it. +Yeah, I totally agree. +Wild sidebar. +I can't believe the internet in its early days was like you described, like no NAT firewalls that stop direct access, no passwords. +We might want to know who you are just so we can assign the files more conveniently to you, you know? +Yeah. +I hearken back to like the bulletin board days where you would dial up into somebody's random computer and you would do stuff in there. +And I hosted a BBS and I interacted with others and it was like, we were all generally operating in good faith because we wanted to kind of play together. +And not until much later did bad actors saying, you know what, I see how I could take advantage of this in a way that suits me and not you. +To which we started to say, all right, well then how do we control for these things? +Today that conversation comes into modern systems development of secure by design, right? +Or a lot of folks will say shift left, right? +Take security into account much earlier into the life cycle as opposed to, oh, we have to tack this on at the end. +So I think the evolution of the internet was necessary for us to get to here. +But as we're seeing newer protocols develop, those are taking this more secure by design approach. +Yeah. +In depth and with layers. +Were you a trade wars fan? +Oh man, that's a name I have not heard in a very long time. +That was a good one though. +Yeah. +I was very much a news and mail kind of relay kind of kid. +Just wanted to see what was going on. +Got very much involved in like understanding how the pretty good privacy would allow you to sign your messages so that way other folks could believe that those were you. +Kind of like a test to truth and that kind of fell apart because again, these are all imperfect systems. +They were, but it was such a world full of possibilities back in those days. +I remember even just sending a mail and getting it back through that whole systems of relays was mind boggling. +At the time I was living on top of a mountain in the middle of Israel and having that ability to connect with other people who there's no way I was ever going to see this variety of people back then. +Like Oh, this opened the world. +Right. +Yeah. +And that kind of fueled my desire to like, okay, what else can I do with these computers with these systems? +And Oh wait, there's this internet thing. +All right. +Well, my mom's going to be ticked off because I'm tying up the phone line for hours and like, all right, well, let's just have some fun. +Yeah. +That's when call waiting was the nemesis. +So I bring, I kind of focus on that a little bit because while we're going to talk about things that are not necessarily positive or people trying to do negative things to something that we all love and has been a very positive thing for the Python ecosystem. +I do want to point out mostly technology is doing really awesome things for people like opening these doors and educating and connecting. +It's just some of the bad people, they like to connect and bad ways. +So before we get too far down that let's, let's just have you give people a quick introduction about yourself. +So, so they all know you. +Hey everyone. +I'm Mike Fiedler. +I'm in New York city and that's where I've been living for the last 15 years, I say, I think. +And I've been working in software development systems engineering for over 30 years across a couple of continents, variety of different companies. +And for the past two years, I think, or three, I've been an active contributor to pypi.org. +Prior to that, I was contributing to a lot of Ruby projects, the chef ecosystem. +And I've worked at a variety of different companies, both startups and enterprises. +You may have heard of some like Datadog, Warby Parker, MongoDB, Capital One, just kind of like working through different scenarios and learning different industries along the way. +For the past year, I've been, well, since January, I've been focusing pretty much purely on pypi.org. +You work for the PSF officially or what's the story? +Yeah. +As of August, I was hired to come on full-time. +We thank you to our grants from Amazon web services, AWS, and some other folks that are chipping in to fund this pypi safety and security role. +But the PSF got some funding and I am the first full-time engineer to focus on pypi.org as a full-time. +In the past, you've spoken to some other folks who were contracted out to build out different aspects or features, but now I'm a full-time maintainer. +Yeah, that's really cool. +You know, the developer in residence at Lukashlanga is playing, working in that role now. +I feel like that was the first one of these types of roles, but now there's a couple, right? +Yeah. +I mean, the PSF is a nonprofit organization, very small staff. +I think we number a total of 12 and of those 12, I think only about five of us are engineers and everything else is volunteer based. +The first developer in residence program, which is Lukash, has been successful enough that we got another organization and grants to fund the security developer in residence, which Seth Larson is doing. +And he is kind of focusing on the wider Python ecosystem as a whole. +Whereas my role is very much more narrowly focused on PyPI.org and the ecosystem surrounding that. +So that way, you know, we can focus on specific targets around security for the packaging world as opposed to the Python core. +Okay. +Well, I do believe if you talk to people about why they like Python and especially why they stick with Python, the language is good. +You can do cool stuff with it, but it's pip install, say your name, say the name of your useful library that just brings so much and makes it so sticky and useful and productive. +And so making sure that we have trust and pip install is really important. +Last year, I think Dustin Ingram came on and talked about some of the stats that he had pulled together that speak about like how much PyPI.org is used. +That doesn't even count for the countless folk out there who are mirroring PyPI packages. +So that way they can have a local cache, you know, deal with corporate firewalls or whatever need, right. +But it's true. +There's the very popular request library or the Django project. +pip install Django, and you have all the things that you need to start a Django project. +Right. +And the speed at which the folks who are kind of working on the tooling like pip or some of the other alternatives out there to enable users to get those packages is such a wonderful tool in anyone's toolbox. +But then very often folks figure out, forget that there is an entire kind of package universe behind what they just did as a consumer. +Right. +So pip install Django is, yeah, I got this thing. +It installed it. +Where did it install it from? +How did it get there? +Who put it up there? +Why is it there? +All of those questions, most people go their entire career with not even having to worry about or think about. +They're just like on the consumer side. +But then on the producer side or the package maintainer or project maintainer, that's, there's a whole other slew of things that one has to worry about. +Yeah. +There's some stuff we'll talk about in there, which will be really fun. +I think also there's the third level of just the people who run IPI and the infrastructure and the stats behind it. +I mean, maybe give us a quick, I kind of started us off down this path. +Maybe give us a quick statement for those who don't necessarily know what PyPI is, but I think more interestingly, maybe try to give us some of the stats about the scale of things behind the. +Sure. +I mean, I haven't, I haven't computed the, the runtime stats in a little bit, but pypi.org stands for the Python package index. +And it's distinct from other things that have pypy in their name, which is a different runtime. +But pypi.org is a package index, very much kind of a, a grocery or a, a store where you would pick up ingredients for the thing that you want to bake, right? +If you wanted to bake a cake, you need your ingredients. +What kind of flour are you going to use? +What kind of sugar? +Sure. +There's different kinds of flour and sugar. +Which one do you want? +How do you know you go and find one and where the package index helps is we store kind of publish all the different kinds of flour and sugar that you might want that other people have spent time developing. +That doesn't mean that there's only one type of flour, but there's a variety and we just make it easy for people to publish their projects. +And as you've highlighted, there's over 480,000 projects live on PyPI right now and over 4.8 or almost 4.9 million releases. +And a release is not a one-to-one to a project. +A project may have many releases. +So for instance, if there's the requests library and they publish a new version that comes as a release. +And then beyond that we have files and files map to releases as you could have a source distribution. +So there's like literally the source code of a given release, or you could have compiled wheels for different platforms. +So there's a lot more files than there are releases and there's a lot more releases than there are projects. +And then on the, like the last stat that we show on the front page is the users. +We do have over 740,000 users on pypi.org. +That doesn't mean that these are active users, but they have at some point signed up for an account on pypi.org. +That's a huge number. +And these are not people who might pip install a thing. +These are people who for some reason or other are interested in potentially creating content for others to use. +Exactly. +Today, the only way you can publish a project on PyPI is by having a user or, you know, it starts with a user. +There's other ways to publish, but you have to have a user to kind of start the process. +And a lot of folks have started to kind of get the idea that if this project needs long-term maintainership, right, it's not just me. +Maybe I should ask somebody else to help co-maintain this. +So it's also not a one-to-one mapping of users to projects or releases or something like that. +For sure. +This portion of Talk Python to Me is brought to you by Sentry. +You know Sentry for their error tracking service. +But did you know you can take that all the way through your multi-tiered and distributed app with their distributed tracing feature? +Distributed tracing is a debugging technique that involves tracking requests of your system starting from the very beginning, like a user action, all the way to the back end, database, and third-party services. +This can help you identify if the cause of an error in one project is due to the error in another. +Every system can benefit from distributed tracing, but they're especially useful for microservices. +In this architecture, logs won't give you the full picture, so you can't debug every request in full just by reading the logs. +Distributed tracing with a platform like Sentry gives you a visual overview about which services were called during the execution of certain requests. +Aside from debugging and visualizing your architecture, distributed tracing also helps you identify performance bottlenecks. +Through a visual like a Gantt chart, you can see if a particular span in your stack took longer than expected and how it could be causing slowdowns in other parts of your app. +Learn more and see some examples in the tracing section at docs.sentry.io to take advantage of all the features of the Sentry platform. +Just create your free account. +And for all of you Talk Python listeners, use the code TALKPYTHON, all one word, and you'll activate a free month of their premium paid features. +Get started today at talkpython.fm slash sentry dash trace. +That link is in your podcast player show notes and the episode page. +Thank you to Sentry for supporting Talk Python to me. +Some of the changes coming, I think, allow for like, almost like a GitHub organization within PyPI, right? +Rather than, well, we're going to create an account and that one account is for all of AWS, for example, which is not really the right granularity, probably. +It definitely isn't, but it historically has been, right? +Like that is just a feature we had never built. +It was never a focus, but over the past year or so, I think we got funded to build out some of the organizations aspect. +We have launched the community organizations. +So that way, if you're running a, an open source project or an ecosystem there, you can sign up today and get an organization name. +We are still working through a long backlog of organizations in order to approve them. +It still requires a, an admin to do so, but we are still working through some of the complexities around corporate organizations when it comes to just as a nonprofit, how can we kind of figure out how to support corporations properly? +Yeah. +I've always thought that that was something of an opportunity to work with corporations more closely on PyPI and indirectly through the PSF. +Your role exists because of these grants, because connections with certain high profile and high consumers of Python tech companies, right? +Like AWS and others. +But there's tons of companies that have things that support their product and they're at least their developers work with and having a way to make them feel more at home on PyPI I think is a good idea. +Beyond what lots of organizations may do is, you know, have some of their in-house engineers contribute to PyPI.org to the warehouse code base. +It's open source. +Everything you're looking at is open source. +That's where I started. +And it's, that's the easiest way of like, Oh, you want this thing? +Open an issue, talk about it with us. +You know, if you want to go ahead and put some effort behind it, we'll welcome that too. +But there is a, a wiki page out there of like packaging fundable improvement projects of like, all right, if you're considering throwing some money at the problem, here are some things we've thought about and would love your assistance with. +And that like there's other ways of just like straight up funding a role that can focus on a particular thing. +Excellent. +All right. +Let's talk about supply chain issues. +We were talking before we went live here that probably the biggest side of security or the biggest, at least from my perspective, what seems like a very huge opportunity for people to do bad things is to just upload malware basically of different ways. +Right? +And I don't want to talk about hacking PyPI org itself or other stuff, but I think that that's probably quite well covered. +And it's more about, can I get trick somebody and through various ways of installing something that they, they didn't. +And that generally falls under the supply chain security side of things. +So I wanted to just point out three examples that just show this is a industry wide problem, not necessarily a PyPI problem, but there is a PyPI manifestation of it. +Right? +Yeah. +So just to lay the groundwork for folks who aren't familiar with supply chain attacks, the notion is that instead of an attacker trying to get onto your computer, they're going to go after something that they have a high probability of knowing is going to be on your computer through for the solar winds as kind of an administrative action. +Well, you know, many, many solar winds were installed on servers, on computers. +That's part of the supply chain that it's not, I'm not going directly after you. +I'm going after something you consume. +Right. +And it can be very, very meta, right? +So one of the examples that I would say that that falls under is this thing called Xcode ghost. +And so I believe this was primarily a Chinese problem, basically because in China, there were a lot of app store developers who weren't either weren't registered as Apple developers or for whatever reason, didn't go, maybe it's just a latency thing. +Didn't go through the app store to get their Xcode or go through the developer portal. +They just found like a local mirror. +What are those local mirrors? +What could go wrong? +I just, I'll just get it from, you know, this IP address instead of apple.com. +Right. +Yeah. +So what it did was it was a backdoored version of Xcode. +So not, they weren't attacking even the things that people were using. +They said, let's take over the developers tool chain. +So whatever they happen to be building, we don't know what that is, but we'll install a virus into their app. +That Apple go in the app store, then whoever installs that app, we'll have it right. +These things get very indirect. +This is kind of the, the challenge is like nobody until somebody surfaced this as an attack, right? +Nobody thought this was a problem. +This is kind of earlier to your comment of like, how do you disprove the evidence, the existence of, of a, of a problem. +And a lot of it is just like, all right, we gotta, we gotta think about every aspect that goes into producing a given piece of software. +But like the strongest answer here is don't download random stuff from people on the internet. +Right. +Like I'm sure that this one in particular had a good reason for having a local mirror, but if you're going to local mirror it, then who is the local mirror and what is there? +What are they doing? +Right. +How, what kind of attestation or assurances do you have that they haven't modified anything in the process? +It's very tricky because I might absolutely trust some company out there that's building a very popular, they have 10 million downloads like that. +Surely that's fine. +But one of their developers or one of their consultants to one of their developers may have, you know, misappropriately gotten their tools. +And it's very hard from the outside to even know that that could be a problem. +So these things are tricky. +Yeah. +I mean, the good news is that there's a large volume of security companies out there who, you know, make their bread and butter by scanning and looking for patterns that looks, you know, sneaky, tricky, and they spend a lot of investigative time digging into these. +We get lots of reports from the, from those types of folk of like, here, this is a new package. +It looks fishy and here's why. +And then we, we take action on those. +I hear you. +Hypo squatting was a big issue for a while. +That's a form of supply chain attack. +Like here, this Xcode ghost is we're going to get people to use a fake Xcode or a broken bad Xcode with that they think is fine. +Right. +Instead of trying to say, take over Django, the package and do some militia to it, try to take over Django or, you know, whatever, right. +Some common misspelling of that and upload that package. +And you could even embed Django. +Right. +And so it still functions. +I don't remember it being spelled this way, but it's working. +So it gotta be fine. +Yeah. +Typo squatting is, is, is very much a prevalent problem. +Right. +And because like, I can't prevent you from making a typo. +Like I literally can't, if you type in Django, that's it game over. +Right. +What I can do is look or receive reports that Django exists. +It looks malware and let's just take that down. +Let's not do that. +Right. +Yeah. +And I was going to say when it comes to type of, oh, you talked about typo scouting and, and I was reminded of a, of, of an article I remember reading around DNS record bit flipping where some computers, some browsers would not properly process a given bit in a memory register for a DNS record. +So this author figured out what those bit flips would be for popular DNS names, registered those DNS names and started just harvesting traffic and said, you know what, this is not anything you can do. +This is just how browsers and memory work. +And that was, I don't know, about six, seven years ago. +And I believe it's been fixed since, but it was like, yeah, there's sometimes there's just not anything that you did wrong. +It's the ecosystem you're in is doing things in a way that you don't expect for something as nefarious as, as like DNS bit flipping. +Like this is where like having outbound firewalls can help a whole lot to say, don't allow traffic that I didn't initiate in some manner. +And if I did have, I have, I initiated the traffic to this address before. +Do you remember zone alarm from the early two thousands? +Yes. +So this is before this is, this harkens back to a slightly less naive version of, I can't believe there was no passwords on the accounts, just on the open internet. +But windows 95, 98, there were no firewalls. +And I, I was at a company that was based inside of a university where we all got ethernet and every computer that plugged in, got its own IP address and all sorts of crazy stuff, but there were no firewalls. +And I remember when that thing came out, I thought, you know what, maybe I'm just gonna go around and put this on all the dev machines. +Like it's kind of insane that we have this incredibly insecure software just on the open internet. +And so I did in all the, when I started, it used to say, do you want to let such and such thing act as a server? +Do you want to let IIS or, you know, NGINX or this type of thing. +Sure, that can be a server. +Then the next pop-up was, do you want to let notepad.exe be a server? +I'm like, huh, that's not probably what it should be doing. +Yeah. +That doesn't sound right. +I said, no. +And then the next one, and the next one, the whole companies and notepad.exe were being servers and I'm like, this can't be good. +And it turned out they had, something had infected it. +And until I put on one of those outbound firewalls, how do you know? +Right. +No one knew there was no indication we had, you know, super fast internet. +It wasn't like it was dragging it down. +I don't even remember what it was doing, but it was bad. +The number one thing that I think we can learn from all of those things is that awareness is the biggest part of security, because if folks aren't aware that downloading something from the internet could be a danger, then they're just going to download it and run it. +If somebody who had a previously version of, you know, software working on their machine suddenly pop up and say, this has been modified. +Are you sure you want to open it? +So many of us just click okay, without reading the dialogue. +It's like, well, wait, think about that for just a second, because you are the biggest kind of enabler and disabler of security, the human behind the keyboard, because you probably have some administrative rights on your computer that allows you to do some stuff. +And in the example with Notepad EXE, I think today, if we were to try to do that on some popular developer environment like VS Code, VS Code does act as a server in a lot of cases. +So it's like, I don't know, should this work as an inbound server or not? +I don't know. +Maybe it's just part of the local language server that I need for autocomplete. +Or maybe it's not. +It's getting more subtle every day. +It is absolutely getting more subtle. +Even Zoom had like a local loopback web server thing, I think for a while. +So before we move off of this typo squatting part of the conversation, out in the audience, we've got a pretty decent question here. +What's the possibility of something like a verified badge for popular packages? +I mean, if Twitter can charge $8 a month. +No, I'm just kidding. +I don't think they're called Twitter anymore. +But the artist formerly known as Twitter. +The challenge there is what does verified mean? +This is something that we kind of introduced some features later on that we'll talk about. +But this notion of verified is like, well, verified by whom? +Where does the level of trust? +Because if a supply chain attack happens for Django, so if you were to like search Django here in PyPI.org and we get Django, all right, we've got Django, the second line Django 425. +And if we were to enter there, like, how do we know? +This is a thing, right? +So I could add a badge here, but that doesn't give me any confidence that any of the Django folk, which are great people, that one of them didn't get compromised and suddenly a new version was pushed. +So verified, I guess it's what does that mean to whom and why? +Because the last thing I want to do is tell people, give them a false sense of security when honestly you're downloading software from the internet. +If you don't have a process to vet what it is you're doing is doing the thing, then you should probably look at that aspect of a, we vetted this version of Django. +We got these hashes, we got these releases, we pin this dependency. +We're happy with this. +And then when you upgrade, you kind of do a similar evaluation. +There's a bunch of projects out there like PyUp and safety and others that will publish, you know, and scan for advisories. +There's also the PyPA advisory database for packages that we know have some problems with them. +So that way you can use other tools to audit what you have installed to see if you have something smelly. +But we are thinking about what it would look like to add a, this release and these files of a given project have been published under, you know, stringent, you know, more secure methods. +Yeah. +I certainly see that a verified wouldn't prove that the Django devs hadn't, you know, somebody could have taken over their computer and swapped out like twine or poetry or whatever they're using to upload the package and do exactly what they did with Xcode Ghost basically. +Right. +Something equivalent to that. +So the last part we want to do is like, we don't want to give people a false sense of security and say, well, PyPI told me this was okay. +And then they find out it wasn't because then that looks really bad for us. +But on the flip side, we are looking at how do we provide mechanisms and measures to publishers to reduce the potential for the situations that you described to happen. +This portion of talk Python to me is brought to you by us over at Talk Python Training. +Let me tell you about one of our really popular courses. +HTMX plus flask modern Python web apps hold the JavaScript. +HTMX is one of the hottest properties in web development today. +And for good reason, you might even remember all the stuff we talked about with Carson Gross back on episode 321. +HTMX along with the libraries and techniques we introduced in our new course, will have you writing the best Python web apps you've ever written clean, fast and interactive all without that front end overhead. +If you're a Python web developer that has wanted to build more dynamic interactive apps, but don't want to or can't write a significant portion of your app in rich front end JavaScript frameworks, you'll absolutely love HTMX. +Check it out over at talk Python dot FM slash HTMX or just click the link in your podcast player show notes. +All right, let me throw some ideas out to you and tell me what I think. +So as I think about this, especially when the very first news a couple years ago, I can't remember exactly the timeframe, but not very long ago, the first malicious PyPI package, you know, NPM had been getting whacked on for a while because JavaScript yellow. +But you know, when it came to PyPI, I was like, okay, this seems to be a little more serious and more pervasive. +And they were often typo squatting type of issues, or people would introduce some package and say, here's a cool thing. +You should check it out. +And it's really a virus or one of those types of things. +And so one of my thoughts, one of the metrics I would have liked, or maybe in the future will like to apply to my local Python environment is don't let me install packages that are too new, or don't let me install install packages that have too few downloads. +And like, give me a mechanism to say that, like, I don't want to ever say pip install something and that something has not existed on PyPI for less than a week. +I don't ever want to be able to say pip install something. +And that thing has less than a thousand or 10,000, whatever downloads unless, and they could say, Nope, you can't install that. +It breaks your rules. +You could say, okay, no, I actually uploaded this. +I really need to, you know, you could do like a pip install of force dash dash force, or you know, some kind of override. +But by default, if I could just say, you know, it has to have at least 5,000 downloads, or I just don't want it. +I feel like at that point, somebody would have discovered, Oh, you know what is actually using a hundred percent CPU usage and crypto mining or whatever it happens to be doing. +I don't want to be the first Guinea pig in the world to discover this. +What do you think about this idea? +The download count one is always an interesting one, right? +It's a topic that comes up a lot. +And like, I can tell you personally from experience that writing a little loop to increase download counts is super easy. +Interesting. +Okay. +Like write a wild true pip install something and like you'll drive up download counts that will be meaningless in the grand scheme of things. +So you could say, well, maybe make it like, it's got to have, you know, a thousand distinct IP addresses, but then, you know, if you own a botnet, then you're good to go. +Okay. +Fair. +This becomes the, like the cat and mouse game of like, all right, well, what is something that is good? +Today we have a mechanism where we don't advertise new packages that have been there for, I think under a week to any kind of crawlers. +So any search engine crawlers. +So if you were going to like Google for Python, Django, and it was a brand new package, you wouldn't find it via Google because we wouldn't advertise that for indexing yet. +Right. +But after a week, like we do. +So that's one method that we have for preventing some of these like newer packages from getting widespread visibility because they, you know, everything is a webpage. +They are all subject to search engine optimization. +Somebody could craft their readme to, you know, be the best hit on Google and therefore they'll show up first. +And with all this crazy AI stuff, it's only getting easier. +Hey ChatGPT, I would like to create a page that is like the Django PyPI page, but I want it to rank highly for this. +Something that we are talking about internally of like, how do we put packages that are brand new, either from some heuristic of a brand new user or a brand new version, or differs enough from the previous versions and kind of put those in kind of a holding or a time out zone to let our security research partners who are really excellent at like just listening to the package feeds and going after and just running all their analysis on them to give them first crack. +Right. +And then when they see, okay, out of these hundred thousand packages that were published in the last 24 hours, 1% need to be addressed or reviewed by a human. +They can raise those red flags and then we can kind of apply the administrative action that is necessary in order to keep the users from getting too much of the bad stuff on their computers. +What about some kind of whitelist or a check back to like sneak or one of these other companies that you kind of referenced there just a moment ago? +Having like published allow lists, right? +These are very prevalent in large corporations that have very strong security policies and they have teams of folks that will maintain internal mirrors of a package index. +So they will disallow any pip install of anything unless you're using their package index. +And I think that is another tool in the security toolbox to have people who are that like security focused to say, we will only allow in the things that we have already tested to be true. +We vetted them and those kind of match our heuristics. +If you scroll down a little bit on the Django page, almost every sidebar to every one of these has these statistics. +This particular one shows GitHub statistics because this package has a GitHub URL, but there's also libraries.io, which is not affiliated with pypi.org. +They're just a really great service and you can search for packages of any shape, kind of any ecosystem, but they have a really good kind of ranking system. +Again, if it works for you, the crux of it, don't install garbage off the internet, right? +Check out what you're doing. +But by using something like libraries, which I don't know why that didn't load. +Probably was just getting a virus. +I probably misspelled it. +Oh yeah. +Just kidding. +But they offer a nice set of stats around a given package. +So you can try and be a little bit more informed on your own. +The challenge there remains that nothing is going to tell you on libraries.io or PyPI if somebody has uploaded malicious software and this is a bad one. +The best we can do is once we know about it, we handle it. +Yeah. +I feel like PyPI has been pretty on top of it. +We try. +I published a blog earlier today where I pulled together a lot of analytics and stats from our inbound malware reporters and it's looking pretty good. +We handle over 80% of inbound reports in under 60 minutes. +I go into the article about the whys and wherefores, the timeliness matters and the response time because the longer something is out there, the worse it can contagion to other folks. +So we try and do as quick as possible, often under like five to 10 minutes, but we also have to do some investigation and kind of like confirm that the report is accurate. +We don't want false positives. +Most of our researchers don't give us false positives. +So shout out to all those folk, but it's hard and time consuming. +I remember one of the more recent IPI supply chain issues where somebody uploaded something bad was attributed to all these different ATP and hacking groups have cutesy names like the solar winds was by something bear. +Hold on. +Which bear? +Cozy bear. +That's the kind of bear it was, which is really Russia state actor hacking. +Right. +And one of the PyPI ones was North Korea. +I think they were doing crypto mining on computers, which seems like a real big waste of I have access to the server in a bank. +But anyway, it works for them. +It works for them, you know, like, but the reason I bring this up is like, it's you all have a serious challenge in that if you're up against state actors from a security perspective, like that's not just script kiddies or some weird automation or, you know, like those are you guys got to be on top of the top of your game. +Right. +This is again, where I think relying on our ecosystem of security partners is so important because they will corroborate intelligence that they've garnered from other ecosystems that are beyond PyPI and be able to identify these kinds of actors. +Me, I see kind of just a slice of what the universe has. +They're going to see a different slice, but broader in spectrum and not necessarily as focused on one particular ecosystem. +So working together, we can kind of do the best that we can for all the users out there. +Excellent. +So let's talk about hypo squatting, which is serious, but also kind of the silliest, kind of not that big of a deal because recommendations could be like, you know, actually use a requirements management system rather than just every time you create a new environment, just type pip install X, Y, and Z. +Like the chances you might fat finger that versus pip install dash R requirements, TXT or, you know, poet something with poetry or whatever. +Right. +So that helps a lot, although it's not perfect. +The other one is more the Xcode go style. +Like what if somebody were to take over one of the other systems and you all had over here, you have a new two factor requirement or high PI. +You want to talk about that? +Yeah, absolutely. +This also was covered on an earlier podcast of talk Python where I think in 2022, we had announced that we were starting to ratchet down the amount of potential. +I think you got the wrong link there. +I do have the wrong link. +Keep going. +It's Dustin, Dustin Ingrams. +Yes, exactly. +I thought I pulled it up. +I put the other one twice. +There we go. +The 2FA story is largely, again, we talked about there's about 740,000 users, right? +These are the publishers of packages, right? +So if in our use case, we talked about Django devs, right? +And I'm sorry to pick on Django. +They're just the one that's up there. +But if one of the Django devs was using a classic problem, which is an email expiry or a domain expiry attack. +So let's say I'm a Django admin maintainer and I use MikeDemand.com as my email address, right? +And that's great. +Because we don't want to ever, we don't use Gmail. +We don't want to use or, you know, the dot me or Outlook. +I'm a good citizen of the internet. +So I got my own domain. +Yeah. +I just haven't been paying attention. +Right. +I haven't been paying attention this year. +Right. +And then let's say I let it expire. +Whoops. +You know, like that happens. +People forget to pay their bills. +Or your credit card gets stolen and canceled. +You forget to renew it there. +And then the other thing goes to spam. +Like it could actually be super easy that that happens. +And it happens all the time, right? +Like people, there are numerous domains that I've registered over the year that I was like, yeah, I don't need that anymore. +Obviously I have never used anything from that domain to sign up for anything securely that's there. +But then someone else can come along and register mikedemand.com, set up an email server, request a password reset, get that email. +And now they can do anything I could have done before. +With 2FA, that entire set of problems goes away. +And we're not even talking about like phishing. +If somebody fishes my password or if they use the same, if I made the mistake and use the same password on two websites and one website stored it in securely, and they pop that in a breach and you know, now they have my username and password. +2FA just solves. +Do you discourage that using the same username and password? +I absolutely discourage that. +I find it very inconvenient to have a separate password. +I just use the letter A. Yeah, that's a choice, right? +It's a bad choice. +No, like the amount of tooling out there today, both free and paid for password management is just so pervasive. +It's almost like irresponsible to not use one. +I 100% agree. +Yeah, I was one password. +I think I don't know if it'll tell me how many I have in here, but I think it's coming up on like 1500 and not quite just just under 1000 different distinct passwords and accounts. +But you know, a lot of people don't want to pay for it. +Bitwarden. +Bitwarden is fantastic. +It's open source. +I don't know if you got a recommendation, but you're right. +It's irresponsible. +I mean, I'm a one password fan. +It's just a great tool. +I used it back when when it was like a single thing and I used it as a as a organization account, right? +Like I was an admin for our org and like managing that lifecycle was pretty sweet. +And then it's like, OK, we have this as an organization. +We have over 400 employees. +Why doesn't everyone have this right now? +So you know, it became a good rollout. +But having a second factor, a two FBA or multifactor MFA, I think, is this notion of something you have versus something, you know. +So let's say that even by using a password manager, you don't know that password anymore, right? +You don't remember it, but let's say you do. +Right. +Like let's say somebody gets your entire vault of passwords. +They still don't have this second factor, which is often a time based one time password or web authentication device, which could be a hardware device or a browser fingerprint. +Like they don't have that. +Right. +It's a defense in depth kind of problem that is solving where it's like you need you need to have two things in order to get through this door. +And if you only have one, that's not good enough. +And using that capability and having that ability on PyPI user management has enabled us to roll out a higher grade of security for the packages and maintainers of those packages by attesting that, well, we know that this maintainer or this publisher of this package has already secured themselves. +So against these kinds of attacks. +Yeah, I can just hear the voices. +In fact, they don't come through an audio form. +They come in email like, you know, on that last episode. +Sometimes they come through on the artist formerly known as Twitter. +Sometimes they come through an email. +But like, you know, Michael, you said that two factor will help you realize you don't seem to realize I'm saying I realize so I don't get this email. +Please don't email me that this doesn't stop phishing. +Like people could still fish you. +You go and they could ask you your name and password and they'll ask for your time based authentication and then they're in. +Yes, that's true. +But it stops some things and stopping some things rather than going, well, it's not good enough. +That is certainly not not a responsible way. +Way to go. +I think it's kind of like making the argument that if nothing is perfect, don't do anything else. +Exactly right. +That's a fallacy. +If you're going to die, don't get out of bed. +Right. +Like, no, like we get out of bed, we go to work, we do our things right. +We ultimately as sad as it is right. +We have an end date. +We hopefully don't know what that is. +But like, do the best you can while you can. +That's where I come to from like, this is the best we know. +Yes. +Will there be something new and exciting tomorrow that is even better? +Maybe. +But until then, let's do the thing that we know to be the best that we can do right now. +Right. +Maybe PASCYs will be awesome. +I don't know about that. +Yeah. +But for example, you know, from a phishing perspective, things like 1Password and Bitwarden have plugins for your browsers and they will suggest to autofill on the right domains. +But if you're on pypi.io, is it a pi.org or, you know, whatever, right. +If they're on some kind of phishing domain, they will not suggest to autofill. +Right. +If you find yourself going to your password manager and going, God, why does this not work? +Like, let me just copy this over. +Stop, figure out why it's not working really, really, really well before you somehow subvert this broken extension that won't autofill. +Right. +There are ways to limit phishing through these mechanisms, even if they're not perfect. +Exactly. +I think I said this before, but like, I'll reiterate it. +You the human are the best defender. +Use your logic, use your sense. +Like don't just click at things mindlessly. +Take a moment, take a look, see that error message. +That looks weird. +Why does that look weird? +The domain I'm on looks a little odd. +The little browser lock symbol isn't locked. +Why is that? +Hmm. +Take a moment. +Notepad.exe once stacked at the server. +Yes, I want to load it. +Come on. +Yes, let it. +I got it. +Yeah. +The reason that I think the news around the 2FA for PyPI.org is not that it exists, but that it's required now. +I think that's what's different since I spoke with Dustin. +We've been on a path and as you've got this blog post open, we've been on a path of like starting with the carrot. +We want to provide as many people in the packaging ecosystem, all the incentive, all the time, all the kind of expectation that they could have in order to set this up voluntarily. +There was even a wonderful giveaway of hardware security keys that Google sponsored, which is excellent. +That doesn't mean you need a physical security key. +You can use them. +You can use software security keys, Google Authenticator or any other tool. +Duo Labs has a nice one, but anything in order to kind of move the bar on this 2FA engagement. +We've seen some decent adoption and it's like, okay, well now let's set a timeline. +This post by Donald kind of starts the clock on that. +We are basically drawing a line in the sand that's saying at the end of 2023, if you want to publish a new package, like that's it, you need to have 2FA. +We've started on that process by requiring 2FA for new users. +So if you registered today, you need to set up 2FA. +Like if you've been around for a while and you don't have it yet, we'll still allow you to upload, but we'll send you a notice that's saying, here's what's going to happen at the end of this year. +And we've slowly been kind of ratcheting down the areas at which 2FA is not required with the intent on basically January, December 31st, January 1st, 2024, enabling the requirement on all accounts. +So that way we can kind of walk away from the problem of, well, I guess one of the Django maintainers got phished and that's why we had a big issue in the ecosystem. +Like I don't want that to be the problem. +But again, apologies to Django, y'all are awesome. +It's because they're so popular and loved that you pick on them, I can tell. +Yes. +Again, this doesn't completely solve all phishing attempts, but it certainly is another layer of defense. +So I think it's certainly worth doing. +Now there was a bit of a pushback. +I think somebody even like rage quit their package temporarily and then said, oh no, I want it back on PyPI when this came out as if it was a big deal. +And this is, you know, this blog post was from May. +The deadline is end of 2023. +In between those two times, GitHub just comes out and goes, everyone gets 2FA right now. +I don't care. +Right. +And it's such a broader, more impactful thing in terms of the many people use Python who are not creating packages, but almost everyone who uses Python is also in some way using GitHub. +And so it just touches so much more of the ecosystem and people are like, oh, okay. +I don't know why there was so much blowback in one and not the other. +But it's an odd thing, right? +Because on the one hand, PyPI or the index itself, right, has been around for about 20 years. +This is a long lived concept in the Python ecosystem of having a place where people can publish software freely, no charge, and others can install that software. +This requirement is a shift, right? +And a lot of folks are like, well, what else is going to happen? +It's like, well, probably nothing, right? +I don't see us talking about other requirements or enforcements unless they're necessary. +Again, I can't predict the future. +And if somebody says that like pass keys are the best way and TOTP is broken and proves it, and the industry wide decides, oh, wow, this is not a good idea. +Let's do this other thing. +Then maybe we'll do that. +But until then, this is the best we've got. +The requirement for 2FA is even on the OWASP top 10 list of why you should be doing this. +And it's like, this is what governments use, companies use, and auditors use to say, we are adhering to the best practices. +Because if you had a security vulnerability reported to your company because you weren't using 2FA, auditors will say, well, why not? +It's in the top 10 list. +It's like the SQL injection of yesteryear. +Yeah. +Just like, just do this, right? +Just solve this class of problem. +You will have other problems. +We all have problems, but solve the ones that we know are relatively easy to solve. +Good advice. +I feel like, you know, when the two factor software problem, like that's not good enough, you know, these YubiKeys and stuff are too tricky. +We're just going to go back to SMS. +Like that's, that's where it's. +I cannot believe that my bank will let me use 2FA. +They forced me to use SMS. +You might want to check out for different banks. +Well, it's like one of the top four banks in the U.S. +It's nuts. +They also have limits on the length, not lower bounds, upper bounds on the length of the password. +My, that, that, that I understand why, right? +Upper bounds. +I understand why, but it usually boils down to like database design and like the cost of doing a database migration. +I hear like, I think it's like 12 or something. +It's very short. +Oh, that's short. +That's way too short. +But here's the thing. +Do you know, it doesn't matter if you have one letter or a hundred letters, the hash is still the same length. +Depending on how you're hashing it. +Yeah. +But they will not be stored. +Like if they're not storing the hash, it makes me extra nervous. +Anyway, onward. +I'm glad they got the SMS 2FA backing it up. +Yeah. +Another thing that I, that I, that I do want to kind of plug on the, the, like the security spectrum and kind of to address the question around like verifiable releases is something that we launched earlier this year, which is called trusted publishers. +That's right. +That's alluded to or linked to in the there we go on our docs.pypi.org of what it is. +Links in the show notes, people can check it out. +Where we leverage an open standard called OpenID Connect. +And today we only implement this with one publishing tool called you know, GitHub actions where the service GitHub actions is now delegated to be a trusted publisher for your project. +When you set this up, you have to opt into this completely. +We didn't do this for you, but you can now opt in to say GitHub actions is allowed to publish my project. +And then you can say, you know what? +None of my humans are allowed to publish the project. +The computer that is getting a short lived token for like five minutes or 10 minutes, whatever it is, is allowed to publish this package and no one else's. +And that's how we can start to build the levels of attestation and kind of the software supply chain security to say, I know where the source code is. +I know the source code that built it. +I know the builder who built it. +I know the builder who published it and no one else tampered with it in the interim. +We're not there to like prove that nobody else tampered, but we are there to say, I can now delegate authority to GitHub, GitHub actions to perform this release for me as opposed to me creating a token in PyPI and giving that token to GitHub actions. +That's how we did it before. +Right. +A long lived permanent token that you put in plain text somewhere, right? +What could go wrong? +I mean, usually like an environment variable or a secrets and GitHub actions, they have pretty good ways of securing data, but again, it's long lived. +So if anything ever happened over there, if anybody dumped a debug log that they shouldn't have, that token could be there. +So by using a trusted publisher flow, you can now have your GitHub actions deployed directly to pypi.org once the artifact is complete and not have to do that token management. +So we're getting short on time, Mike. +What else do you want people to know about what you all in particular, what you're doing at PyPI and some of the initiatives and maybe how they can help? +The top of mind for me right now is the malware reporting project that we're engaged in. +And that's kind of linked to at the very bottom of my blog from today, the inbound malware reporting blog, where we are looking to establish what a kind of machine readable protocol would be to interact with security researchers. +A few of them have chimed in already on what they think of, and we're just kind of building the conversation around what it would look like to report. +How do you like to report? +And then we'll proceed with whatever guidance we get there and kind of build out the payloads and stuff like that all the way at the bottom, very bottom, all the way at the bottom. +And once we have this format in place, we're going to be building out like the infrastructure and ecosystem in order to submit those payloads and then figure out how to kind of put packages in timeout while these payloads are being investigated. +So that way we can continue to provide a secure ecosystem for all users of pypi.org. +I think that's great. +I certainly, you know, these companies that are checking out and just monitoring the flow of packages and scanning them, that's a huge service. +Is there, there probably is, never will be like a bug bounty equivalent. +Is there? +I mean, never say never, but. +Never say never. +From that perspective, it becomes a bit of a challenge because then you could start funneling money through a bug bounty program because we are offering an ability for people to create packages and then saying, we're giving you a monetary incentive to report them to us. +So it's like, well, now we've given you a pipeline for money. +There's a whole shadow industry of like, you first create it, then you get it popular, then you report it. +Yeah, yeah, yeah. +No, I hear you. +Yeah. +But you know, no, no ideas too farfetched. +We like talking about ideas and figuring out what, what makes sense and kind of, again, with a lot of security work is like, okay, well, how can this go wrong? +How can this fail? +Right. +How can it be gamed? +Yeah, absolutely. +Well, I, for one, feel better that you're putting all your time and energy into focusing on these problems and seeing how we can make PyPI better for everyone. +Almost everyone, not for everyone, but for 99.9% of us, for most people, just want to use it in a solid way to build Python software. +That's kind of why I was drawn to it, right? +Like to contributing to it is such a foundational piece of modern day infrastructure that it's important that it be safe, secure, convenient, useful to anybody who wants to use it because Python itself is such a ubiquitous language across the planet and beyond that, you know, we want to make it the right thing. +Yeah. +Surprisingly, every time you say that statement, it's more true. +Like that just, that graph continues to go up and in surprising ways. +All right. +Before you get out of here, I'll ask you one of the final questions, a notable PyPI package, not malware ridden, but a good, useful one. +What do you recommend? +Anything you come across that's awesome lately? +I'm a huge fan of pytest and I know that, you know, your big pals with Brian Okken, hey Brian, who talks a lot about testing and pytest plugins are a wonderful extension to pytest. +Yes. +And there's so many of them out there and there's even like an awesome pytest aggregator of these. +And I think I have one on here, which is called pytest Socket. +Nice. +Which I maintain till today. +But the one that I want to point out is one that I recently learned about, which is called ICDiff. +I, the letter C, diff. +I don't even know if it's on this. +It's the letter C. +I gotcha. +Yeah, there it is. +So that's not the pytest package, but there's an extension pytest ICDiff. +We'll get there. +So this uses that other one. +But the notion here is a lot of times you get big pytest output if you're comparing, you know, dictionaries, lists or stuff that has lots of data. +Sometimes detecting the difference is very hard in the terminal and the pytest ICDiff extension will help highlight a lot of these with colors, with spacing, which makes finding the problem much easier. +Yeah. +That seems super helpful right there. +And it does a partial character by character diff and line by line diff with different colors. +Yeah. +And the here's what we expected. +Here's what you got. +Yeah. +So I'm learning that there's even more madness to the pretty print. +So it could say from pprint import pprint, but there's also apparently a pprint pp with ppi. +Okay. +Yeah. +I don't know. +That's more things to explore. +It's always, it's going to be in one of those 400,000 packages on PyPI. +Right. +It's got to be there. +And it might be a little different. +It might be just enough different to meet this use case that is, you know, perfect. +Yeah. +So pprint plus plus that's what the PP is like CPP up there. +Okay. +Got it. +Notepad plus plus dot exe. +It wants to act as a server. +All right. +Let's leave it with that. +I guess a final thing, people are excited to hear about this. +They want to get engaged. +You know, they have ideas. +They want to reach out to you. +What do you say? +Open an issue for us on, you know, the warehouse repository, if it's relevant to the warehouse code base. +If you need to reach me directly, I'm on GitHub as Mike the man. +I'm on a mastodon as Mike the man at hackyderm.io. +Or if all of that fails, go ahead and email me at Mike at python.org. +Awesome. +Thank you so much. +Thanks for being on the show and giving us a status report here. +Absolutely. +Thanks for having me, Michael. +This has been another episode of talk Python to me. +Thank you to our sponsors. +Be sure to check out what they're offering. +It really helps support the show. +Take some stress out of your life. +Get notified immediately about errors and performance issues in your web or mobile applications with Sentry. +Just visit talkpython.fm slash Sentry and get started for free. +Be sure to use the promo code talk Python, all one word. +Want to level up your Python? +We have one of the largest catalogs of Python video courses over at talk Python. +Our content ranges from true beginners to deeply advanced topics like memory and async. +And best of all, there's not a subscription in sight. +Check it out for yourself at training.talkpython.fm. +Be sure to subscribe to the show. +Open your favorite podcast app and search for Python. +We should be right at the top. +You can also find the iTunes feed at slash iTunes, the Google Play feed at slash play and the direct RSS feed at slash RSS on talkpython.fm. +We're live streaming most of our recordings these days. +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm slash YouTube. +This is your host, Michael Kennedy. +Thanks so much for listening. +I really appreciate it. +Now get out there and write some Python code. +Bye. +Thank you for watching. +Please subscribe to my channel. +you diff --git a/flow_results/file_lengths.json b/flow_results/file_lengths.json index 56364dc..3673fc4 100644 --- a/flow_results/file_lengths.json +++ b/flow_results/file_lengths.json @@ -409,5 +409,7 @@ "430-gradio.txt": 16128, "431-visualizing-cpython-release.txt": 14297, "432-beanie-pydantic-upgrade.txt": 11557, - "433-litestar.txt": 13940 + "433-litestar.txt": 13940, + "434-mobile-apps.txt": 13837, + "435-pypi-security.txt": 13378 } \ No newline at end of file diff --git a/flow_results/transcript_filenames.txt b/flow_results/transcript_filenames.txt index 186ea9e..5a15ef8 100644 --- a/flow_results/transcript_filenames.txt +++ b/flow_results/transcript_filenames.txt @@ -431,3 +431,5 @@ 431-visualizing-cpython-release.txt 432-beanie-pydantic-upgrade.txt 433-litestar.txt +434-mobile-apps.txt +435-pypi-security.txt diff --git a/talk-python-transcripts b/talk-python-transcripts index 56644e8..4b69d68 160000 --- a/talk-python-transcripts +++ b/talk-python-transcripts @@ -1 +1 @@ -Subproject commit 56644e8afd2461c38f9318dae205c25b8caccfc1 +Subproject commit 4b69d680e0d4ea1f2a32496120972e34128551b0