Executing From Strings

00:00 In the previous lesson, I showed you how to examine an object using the built-in functions id(), dir(), and vars().

00:07 I always forget which of those gets the s on the end, 50-50 whether I get it right the first time. In this lesson, I’m going to show you how to accidentally give malicious users control over your machine and wipe out your hard drive.

00:19 Before getting started on this feature, I really need to highlight that it should only be used rarely and in very particular situations. You have to be very careful with the three functions I’m covering in this lesson.

00:31 Seriously, you shouldn’t do this. Yeah, I know I’m getting silly, but you got to be careful.

00:37 Why all this caution? Well, Python is able to evaluate strings as Python programs. That means you can store any program you want inside of a string and execute it as if it was code.

00:48 The tools for doing this, should you decide to ignore my warnings, are eval() for evaluating expressions, exec() for running arbitrary Python, and compile() for compiling an object that you could then pass to eval() or exec() so you can shoot yourself in the foot more efficiently.

01:06 That’s it, roll your eyes, but at least you can’t say I didn’t warn you. One of the particularly tricky parts of this is there are ways of obfuscating your code.

01:15 For example, you might think you could scan any data for the word subprocess and eliminate it, saving you from one avenue of doom. The problem is there are ways of dynamically loading modules using encoded descriptions so that the word subprocess wouldn’t have to be involved and that module could still be loaded, and then you could use it to call rm and wipe out your hard drive. I think I’ve made it scary enough.

01:39 Nothing but clowns and spiders here on in. Okay, let’s go see what you shouldn’t do. An expression in Python is a combination of objects and operators that returns a value.

01:50 Not all code is an expression, but all expressions are code. The rules here are similar to those of defining lambdas. Let’s start with a short expression.

02:03 There’s my string containing a Python expression.

02:08 And there, I’ve executed it as Python, resulting in the answer being returned. By default, eval() uses the existing local and global scopes, but you can optionally pass in dictionaries to set them specifically.

02:28 That first dictionary is the global scope, which I’ve left empty. The second is the local scope. You almost never need both, but if you’re trying to limit what is in the scope, you could pass in empty dictionaries for both, ensuring that the expression doesn’t have access to the actual global and local scopes outside of the call. That doesn’t prevent somebody from doing something malicious inside of the code, though.

02:51 For a slightly safer tool, the ast module has a function called literal_eval(). This evaluates strings, bytes, numbers, tuples, lists, dicts, sets, and Booleans. It can’t do full expressions and so is less problematic.

03:06 But that makes it less useful. Even then, there are ways for malicious code to crash your program or to cause a denial of service by spinning your CPU. There are also third-party libraries that will do code parsing, giving you an abstract syntax tree.

03:20 For example, if you wanted to build a command-line calculator that supported arbitrary math, you could define a mini-language that looked a lot like Python expressions, parse it with the library, then evaluate it.

03:31 Anything not recognized by your parser would get rejected, so unless you built tools for erasing your hard drive as expressions, that would be safer. eval() is dangerous, but there is worse.

03:43 Let’s look at its more flexible, and more problematic, cousin, exec().

03:48 Another code string,

03:50 this time defining a

03:55 function. Note that this string doesn’t actually run the function. It could, but this one doesn’t. When I exec() the string that defines the function, the function becomes defined in my space.

04:07 And now I can run it like any other function. And this is one of the many reasons exec() is dangerous. You can redefine a function in Python, overriding an existing one, replacing whatever was there with whatever you want.

04:20 Then, when your program went to call the original, it would call the replacement instead. If you pass exec() or eval() a string, each time you call it, Python turns it into bytecode before running it.

04:32 If you’re running a string over and over again, say you just don’t like your toes and want to pick them off with a revolver one at a time, well, then, you could compile the string, giving you a more efficient way of reducing your shoe size.

04:48 The call to compile() takes a number of arguments. The first is the thing being compiled, so that would be a code string like you would pass to eval() or exec().

04:57 The second argument is the name of the file from which the code came. I believe this is connected to its module representation under the covers. The convention is to use <string> when compiling a string, like I have.

05:11 The third argument is whether the compile target is for the eval() function or the exec() function. Essentially, which rules do you want?

05:18 Evaluate an expression or a full chunk of code?

05:23 The result of the call is a code object, which you then can pass to eval() or exec(), respectively. For the sake of brevity, I skipped over a couple of the optional arguments to compile().

05:37 Let me remedy that fact. You already saw the source and filename args, and you saw mode, and I mentioned two examples, exec and eval.

05:47 There is a third one, though, single, which is kind of between the two. It allows for code like exec does, but only a single line at a time.

05:56 This tends to get used for things like implementing a REPL where you might want to restrict multiline input. The next two are messy and seldom used. The flags and dont_inherit arguments control how the compiler behaves.

06:09 For example, you can set the PyCF_TYPE_COMMENTS flag to enable using comments to specify type information on your statements. You can also control what future features are turned on. Python sometimes implements features into a release for testing before they’re fully supported or backports features into older versions to give future compatibility. You then get at said features by importing them from __future__.

06:35 For example, Python 3 changed print from a statement to a function. In later versions of Python 2, you could import print as a function from __future__. So flags and dont_inherit allow you to control this behavior.

06:52 Finally, Python is able to optimize the generated code. Setting this level high enough removes certain kinds of debug calls, assertions, and docstrings for more efficiency.

07:03 There’s a full course and tutorial on this, so if you’ve survived this far with your toes intact and want to try again, you could roll the dice one more time.

07:12 How’s that for a mixed metaphor?

07:15 There are so many built-in functions that some are hard to group. So next up, I’ll finish off what I started with the stragglers.