Brian Overland, John Bennett

June 2019

Addison Wesley Professional

672 Pages

# Supercharged Python: Take Your Code to the Next Level

Tapping into the full power of Python doesn’t have to be difficult. Supercharged Python is written for people who’ve learned the fundamentals of the language but want to take their skills to the next level.

After a quick review of Python, the book covers: advanced list and string techniques; all the ways to handle text and binary files; financial applications; advanced techniques for writing classes; generators and decorators; and how to master packages such as Numpy (Numeric Python) to supercharge your applications!- Use profilers and “magic methods” to code like a pro
- Harness the power of regular expressions to process text quickly with a single statement
- Take advantage of 22 coding shortcuts, along with performance tips, to save time and optimize your code
- Create really useful classes and objects, for games, simulations, money, mathematics, and more
- Use multiple modules to build powerful apps while avoiding the “gotchas”
- Import packages to dramatically speed up statistical operations–by as much as 100 times!
- Refer to the five-part language reference to look up fine points of the language

- Supercharged Python demonstrates techniques that allow you to write faster and more powerful code, whether you’re manipulating large amounts of data or building sophisticated applications.

Brian Overland is a textbook author, sometime actor, programmer, film reviewer, and novelist. He has been professionally programming with the C family of languages since the early 1980s and spent 10 years at Microsoft, first as a software tester and then as programmer/writer, manager, and project lead. Almost unique among programmers, he is an award-winning writer deeply committed to teaching and simplifying advanced concepts. You can read his comments on technology, reviews, and his upcoming book projects at brianoverland.com.

Table of Contents

Preface xxiii

Acknowledgments xxvii

About the Authors xxix

Chapter 1: Review of the Fundamentals 1

1.1 Python Quick Start 1

1.2 Variables and Naming Names 4

1.3 Combined Assignment Operators 4

1.4 Summary of Python Arithmetic Operators 5

1.5 Elementary Data Types: Integer and Floating Point 6

1.6 Basic Input and Output 7

1.7 Function Definitions 9

1.8 The Python “if” Statement 11

1.9 The Python “while” Statement 12

1.10 A Couple of Cool Little Apps 14

1.11 Summary of Python Boolean Operators 15

1.12 Function Arguments and Return Values 16

1.13 The Forward Reference Problem 19

1.14 Python Strings 19

1.15 Python Lists (and a Cool Sorting App) 21

1.16 The “for” Statement and Ranges 23

1.17 Tuples 25

1.18 Dictionaries 26

1.19 Sets 28

1.20 Global and Local Variables 29

Summary 31

Review Questions 31

Suggested Problems 32

Chapter 2: Advanced String Capabilities 33

2.1 Strings Are Immutable 33

2.2 Numeric Conversions, Including Binary 34

2.3 String Operators (+, =, *, >, etc.) 36

2.4 Indexing and Slicing 39

2.5 Single-Character Functions (Character Codes) 42

2.6 Building Strings Using “join” 44

2.7 Important String Functions 46

2.8 Binary, Hex, and Octal Conversion Functions 47

2.9 Simple Boolean (“is”) Methods 48

2.10 Case Conversion Methods 49

2.11 Search-and-Replace Methods 50

2.12 Breaking Up Input Using “split” 53

2.13 Stripping 54

2.14 Justification Methods 55

Summary 56

Review Questions 57

Suggested Problems 57

Chapter 3: Advanced List Capabilities 59

3.1 Creating and Using Python Lists 59

3.2 Copying Lists Versus Copying List Variables 61

3.3 Indexing 61

3.4 Getting Data from Slices 64

3.5 Assigning into Slices 67

3.6 List Operators 67

3.7 Shallow Versus Deep Copying 69

3.8 List Functions 71

3.9 List Methods: Modifying a List 73

3.10 List Methods: Getting Information on Contents 75

3.11 List Methods: Reorganizing 75

3.12 Lists as Stacks: RPN Application 78

3.13 The “reduce” Function 81

3.14 Lambda Functions 83

3.15 List Comprehension 84

3.16 Dictionary and Set Comprehension 87

3.17 Passing Arguments Through a List 89

3.18 Multidimensional Lists 90

Summary 93

Review Questions 93

Suggested Problems 94

Chapter 4: Shortcuts, Command Line, and Packages 95

4.1 Overview 95

4.2 Twenty-Two Programming Shortcuts 95

4.3 Running Python from the Command Line 115

4.4 Writing and Using Doc Strings 117

4.5 Importing Packages 119

4.6 A Guided Tour of Python Packages 121

4.7 Functions as First-Class Objects 123

4.8 Variable-Length Argument Lists 125

4.9 Decorators and Function Profilers 128

4.10 Generators 132

4.11 Accessing Command-Line Arguments 138

Summary 141

Questions for Review 142

Suggested Problems 142

Chapter 5: Formatting Text Precisely 145

5.1 Formatting with the Percent Sign Operator (%) 145

5.2 Percent Sign (%) Format Specifiers 147

5.3 Percent Sign (%) Variable-Length Print Fields 150

5.4 The Global “format” Function 152

5.5 Introduction to the “format” Method 156

5.6 Ordering by Position (Name or Number) 158

5.7 “Repr” Versus String Conversion 161

5.8 The “spec” Field of the “format” Function and Method 162

5.9 Variable-Size Fields 176

Summary 178

Review Questions 179

Suggested Problems 179

Chapter 6: Regular Expressions, Part I 181

6.1 Introduction to Regular Expressions 181

6.2 A Practical Example: Phone Numbers 183

6.3 Refining Matches 185

6.4 How Regular Expressions Work: Compiling Versus Running 188

6.5 Ignoring Case, and Other Function Flags 192

6.6 Regular Expressions: Basic Syntax Summary 193

6.7 A Practical Regular-Expression Example 200

6.8 Using the Match Object 203

6.9 Searching a String for Patterns 205

6.10 Iterative Searching (“findall”) 206

6.11 The “findall” Method and the Grouping Problem 208

6.12 Searching for Repeated Patterns 210

6.13 Replacing Text 211

Summary 213

Review Questions 213

Suggested Problems 214

Chapter 7: Regular Expressions, Part II 215

7.1 Summary of Advanced RegEx Grammar 215

7.2 Noncapture Groups 217

7.3 Greedy Versus Non-Greedy Matching 219

7.4 The Look-Ahead Feature 224

7.5 Checking Multiple Patterns (Look-Ahead) 227

7.6 Negative Look-Ahead 229

7.7 Named Groups 231

7.8 The “re.split” Function 234

7.9 The Scanner Class and the RPN Project 236

7.10 RPN: Doing Even More with Scanner 239

Summary 243

Review Questions 243

Suggested Problems 244

Chapter 8: Text and Binary Files 245

8.1 Two Kinds of Files: Text and Binary 245

8.2 Approaches to Binary Files: A Summary 247

8.3 The File/Directory System 248

8.4 Handling File-Opening Exceptions 249

8.5 Using the “with” Keyword 252

8.6 Summary of Read/Write Operations 252

8.7 Text File Operations in Depth 254

8.8 Using the File Pointer (“seek”) 257

8.9 Reading Text into the RPN Project 258

8.10 Direct Binary Read/Write 268

8.11 Converting Data to Fixed-Length Fields (“struct”) 269

8.12 Using the Pickling Package 278

8.13 Using the “shelve” Package 280

Summary 282

Review Questions 283

Suggested Problems 283

Chapter 9: Classes and Magic Methods 285

9.1 Classes and Objects: Basic Syntax 285

9.2 More About Instance Variables 287

9.3 The “_ _init_ _” and “_ _new_ _” Methods 288

9.4 Classes and the Forward Reference Problem 289

9.5 Methods Generally 290

9.6 Public and Private Variables and Methods 292

9.7 Inheritance 293

9.8 Multiple Inheritance 294

9.9 Magic Methods, Summarized 295

9.10 Magic Methods in Detail 297

9.11 Supporting Multiple Argument Types 320

9.12 Setting and Getting Attributes Dynamically 322

Summary 323

Review Questions 324

Suggested Problems 325

Chapter 10: Decimal, Money, and Other Classes 327

10.1 Overview of Numeric Classes 327

10.2 Limitations of Floating-Point Format 328

10.3 Introducing the Decimal Class 329

10.4 Special Operations on Decimal Objects 332

10.5 A Decimal Class Application 335

10.6 Designing a Money Class 336

10.7 Writing the Basic Money Class (Containment) 337

10.8 Displaying Money Objects (“_ _str_ _”, “_ _repr_ _”) 338

10.9 Other Monetary Operations 339

10.10 Demo: A Money Calculator 342

10.11 Setting the Default Currency 345

10.12 Money and Inheritance 347

10.13 The Fraction Class 349

10.14 The Complex Class 353

Summary 357

Review Questions 357

Suggested Problems 358

Chapter 11: The Random and Math Packages 359

11.1 Overview of the Random Package 359

11.2 A Tour of Random Functions 360

11.3 Testing Random Behavior 361

11.4 A Random-Integer Game 363

11.5 Creating a Deck Object 365

11.6 Adding Pictograms to the Deck 368

11.7 Charting a Normal Distribution 370

11.8 Writing Your Own Random-Number Generator 374

11.9 Overview of the Math Package 376

11.10 A Tour of Math Package Functions 376

11.11 Using Special Values (pi) 377

11.12 Trig Functions: Height of a Tree 378

11.13 Logarithms: Number Guessing Revisited 381

Summary 385

Review Questions 385

Suggested Problems 386

Chapter 12: The “numpy” (Numeric Python) Package 387

12.1 Overview of the “array,” “numpy,” and “matplotlib” Packages 387

12.2 Using the “array” Package 388

12.3 Downloading and Importing “numpy” 390

12.4 Introduction to “numpy”: Sum 1 to 1 Million 391

12.5 Creating “numpy” Arrays 392

12.6 Example: Creating a Multiplication Table 405

12.7 Batch Operations on “numpy” Arrays 406

12.8 Ordering a Slice of “numpy” 410

12.9 Multidimensional Slicing 412

12.10 Boolean Arrays: Mask Out That “numpy”! 415

12.11 “numpy” and the Sieve of Eratosthenes 417

12.12 Getting “numpy” Stats (Standard Deviation) 419

12.13 Getting Data on “numpy” Rows and Columns 424

Summary 429

Review Questions 429

Suggested Problems 430

Chapter 13: Advanced Uses of “numpy” 431

13.1 Advanced Math Operations with “numpy” 431

13.2 Downloading “matplotlib” 434

13.3 Plotting Lines with “numpy” and “matplotlib” 435

13.4 Plotting More Than One Line 441

13.5 Plotting Compound Interest 444

13.6 Creating Histograms with “matplotlib” 446

13.7 Circles and the Aspect Ratio 452

13.8 Creating Pie Charts 455

13.9 Doing Linear Algebra with “numpy” 456

13.10 Three-Dimensional Plotting 463

13.11 “numpy” Financial Applications 464

13.12 Adjusting Axes with “xticks” and “yticks” 467

13.13 “numpy” Mixed-Data Records 469

13.14 Reading and Writing “numpy” Data from Files 471

Summary 475

Review Questions 475

Suggested Problems 476

Chapter 14: Multiple Modules and the RPN Example 477

14.1 Overview of Modules in Python 477

14.2 Simple Two-Module Example 478

14.3 Variations on the “import” Statement 482

14.4 Using the “_ _all_ _” Symbol 484

14.5 Public and Private Module Variables 487

14.6 The Main Module and “_ _main_ _” 488

14.7 Gotcha! Problems with Mutual Importing 490

14.8 RPN Example: Breaking into Two Modules 493

14.9 RPN Example: Adding I/O Directives 496

14.10 Further Changes to the RPN Example 499

14.11 RPN: Putting It All Together 508

Summary 513

Review Questions 514

Suggested Problems 514

Chapter 15: Getting Financial Data off the Internet 517

15.1 Plan of This Chapter 517

15.2 Introducing the Pandas Package 518

15.3 “stock_load”: A Simple Data Reader 519

15.4 Producing a Simple Stock Chart 521

15.5 Adding a Title and Legend 524

15.6 Writing a “makeplot” Function (Refactoring) 525

15.7 Graphing Two Stocks Together 527

15.8 Variations: Graphing Other Data 530

15.9 Limiting the Time Period 534

15.10 Split Charts: Subplot the Volume 536

15.11 Adding a Moving-Average Line 538

15.12 Giving Choices to the User 540

Summary 544

Review Questions 545

Suggested Problems 545

Appendix A Python Operator Precedence Table 547

Appendix B Built-In Python Functions 549

Appendix C Set Methods 577

Appendix D Dictionary Methods 583

Appendix E Statement Reference 587

Variables and Assignments 587

Spacing Issues in Python 589

Alphabetical Statement Reference 590Addison Wesley Professional