Python on Unix and Linux System Administrator's Guide

(see also functions) regular expressions, 88–97 reusing code, 16–20 raw strings in, 91 Python community, 5, 21 rehash command, 37–38 Python eggs rehashx command, 38 changing standalone .py file into, 264 relational serialization, 376–384 defined, 266 remote procedure calls (RPCs), 158–164 for package management, 258, 266–270 Pyro framework, 161–164 installing on filesystems, 262 XML-RPC, 158–161 Python Package Index, 271–272 remove( ) method (os module), 190 Python packages, 23 removing (see deleting) Python prompts, versus IPython prompts, 26 renaming files in directory tree, 194 Python Remote Objects (see Pyro framework) rep function, 66–69 Python Standard Library, 5 replace( ) method, 85 Python wrappers (see wrappers) reporting and documentation, 123–145 python-ldap module, 407 automated information gathering, 123–126 python-reportlab package, 139 receiving email, 125–126 python-textile package, 134 information distribution, 141–145 sending email, 141–144 Q sending email attachments, 143 question mark (?) information formatting, 135–141 saving as PDF files, 138–141 to obtain object information, 54 manual information gathering, 126–135 to search for objects, 56 ReportLab library, 138 ?? to obtain object information, 54 __repr__ representation of strings, 75 question mark (?) for help, 12, 31 request( ) method (httplib module), 154 queues, threading, 305 reset function, 65 %quickref command, 32 ReST (reStructuredText) format, 131, 271 quotation marks ReSTless utility, 271 creating strings with, 72 restoring data, 199–204 results history, 62–64 R retrbinary( ) method (ftpmodule), 156 raw strings, 73 return codes, with Subprocess, 290–298 in regular expressions, 91 reusing code, 16–20 re module, 88 rstrip( ) method, 79 (see also regular expressions) rsync, 415 re-imaging partitions, 242 rsync utility, 195–197 read( ) method, 102 run function, 66 readability of Python, 2 running shell commands, 9 (see also whitespace) reading files, 102–104 S reading input (see input) s attribute, 47 readline support, 59 save function, 66 readline( ) method, 103 SAX (simple API for XML), 116 readlines( ) method, 103 Scapy rec directive, 50 creating scripts with, 175–176 recv( ) function (socket module), 148 Scapy program, 173–175 Red Hat systems administration, 245 scheduling process, 316–317 registering packages with Python Index, 271– screen application, 300–301 272 script wrappers (see wrappers) 430 | Index

scripts (see modules) cPickle library, 362–363 search( ) method, 95 pickle module, 357–362 search( ) method (imaplib), 125 shelve module, 363–366 searching YAML data format, 366 command history, 61 ZODB module, 370–376 for objects by name, 55–57 simplicity of Python, 1, 6–266 within strings, 75–79, 85 single quotation mark (') (see also substrings) creating strings with, 72 using regular expressions, 88–97 slicing strings, 78 searching (finding) (see also strings; substrings) for duplicates in directory trees, 188 SMTP authentication, 142 TAR file contents, 201–204 smtplib package, 141 using pattern matching, 193–195 SNMP, 205–226 searching strings device control, 224–225 for substrings discovering data centers, 211–214 and replacing them, 85 enterprise integration with Zenoss, 207, serialization 225–226 relational, 376–384 extending Net-SNMP, 222–224 SQLAlchemy ORM, 382–385 hybrid SNMP tools, creating, 220–222 SQLite library, 376–379 Net-SNMP, about, 208–211 Storm ORM, 379–382 overview of, 205–208 simple, 357–376 installation and configuration, 206–208 cPickle library, 362–363 retrieving multiple values, 214–220 pickle module, 357–362 snmpstatus tool, 220 shelve module, 363–366 socket( ) function (socket module), 147 YAML data format, 366 socket module, 147–153 ZODB module, 370–376 Solaris systems administration, 245 Server class (Django), 346 source code, obtaining, 54 Service class (Django), 346 span( ) method, 96 setuptools, 258 spawning processes from command-line tool, Easy Install module 401 advanced features, 261–266 split( ) method, 81–83 entry points and console scripts, 270–271 splitlines( ) method, 83 SFTP (Secure FTP), 166 SQLAlchemy, 197 sh profile, 48–50 SQLAlchemy ORM, 382–385, 382–385 shadow history, 61 SQLite library, 376–379, 376–379 shell (see IPython shell) ssh keys, 231 shell commands, integrating with command- SSH protocol, 164–167 line tools, 397–402 SSL (Secure Sockets Layer), 143 shell execute, 36 standard input and output, 105–108 shell prompt, numbering of, 25 standard library, 5 shelve module, 124, 363–366 standard out, suppressing, 290 shortcuts, with IPython shell, 64–69 start( ) method, 96 showPage( ) method (ReportLab), 140 startswith( ) method, 78 shutil module, 179 starttls( ) method, 143 copying data tree (example), 179 statement execution, 8–12 deleting data tree (example), 181 store function, 65 moving data tree (example), 180 Storm ORM, 379–382, 379–382 simple serialization, 357–376 __str__ representation of strings, 75 Index | 431

str type, 71 searching TAR files, 201–204 (see also strings) text attribute (Element object), 118 string input (see input) text files, working with, 100–105 string output (see output) (see also input) string processing with Unix shell, 44–48 (see also output) string representations, 25 creating files, 100–102 stringIO module, 108 log parsing (example), 110–116 strings, 71 parsing XML with ElementTree, 116–120 converting case of, 81 reading files, 102–104 creating (str type), 72–75 writing files, 104–105 creating (Unicode), 85–87 text strings, 75 (see strings) extracting data from (see also substrings) looking within strings, 75–79 textile, 134–135 hacking Apache config file (example), 97– third-party packages, 5 100 threading timer, 310 joining together (concatenating), 84 threads, 301–313 processing as input, 108 ticket tracking system (Trac), 144 raw strings, 73, 91 time utility (Unix), 90 removing content from, 79–81 timeit utility, 89 replacing substrings within, 85 timer for threads, 310 searching within, 75–79 Trac wiki, 144 using regular expressions, 88–97 tracking system (Trac), 144 splitting at delimiter, 81–83 trailing whitespace, stripping from strings, 79 unicode strings, 85–87 triple quotes for string creation, 73 strip( ) method, 79–81 true/false usage pattern (optparse), 391 subprocess module, 11 Twisted framework, 167–173 subprocess.call, 10 TwistedSNMP library, 206 Subprocess module, 289–298 using return codes, 290–298 U substrings, 75 (see also strings) Ubuntu administration, 245 joining (concatenating), 84 UDP ports for SNMP, 205 replacing within strings, 85 underscore (_) searching strings for, 75–85 for results history, 62–64 Sun Solaris systems administration, 245 __ (in variable names), 38 Supervisor utility, 298–300 __ object, 57 synchronizing directories with rsync, 195–197 ___ object, 57 sys module, 105–108 unicode strings, 85–87 sys.argv, 388 Unix mashups, 397–402 sysDescr OID, 206, 209 Unix programming, cross-platform, 228–238 system_profiler utility, 119 Unix shell, working with, 34–50, 39–41 alias function, 34 T bookmark command, 41 dhist command, 42 %-TAB, 31 pwd command, 43 tab complete, 11 rehash command, 37 tab completion, 28 rehashx command, 38 tabbing code (see indenting Python code) sh profile, 48–50 tag attribute (Element object), 118 shell execute, 36 tar package, 199–204 string processing, 44–48 432 | Index

variable expansion, 43 writing files, 104–105 upgrading packages, 262 writing output (see output) upper( ) method, 81 uppercase, converting entire string to, 81 X URL-based content, processing as input, 109 urllib module, 109, 157–158 XHTML, converting Textile to, 134 urllib2 module, 158 XML file parsing with ElementTree, 116–120 XML-RPC, 158–161 V XML-RPC, communicating with Zenoss via, 226 Vainio, Ville, 22 variable expansion, 43 Y variables deleting from interactive namespace, 65 YAML data format, 366–370 starting with __ (double underscore), 38 VBox (PyGTK), 329 Z vertical box (PyGTX), 329 zendmd API, 226 Vim, 209 Zenoss API, 207, 225–226 virtual machine, creating, 237 managing Windows servers from Linux, virtualenv tool, 279–283 253–256 Virtualization, 246 ZODB module, 370–376 W walking MIB trees, 210 web applications, building, 334–335 web server port monitoring (example), 148– 153 Web-based content, processing as input, 109 web2ldap tool, 408 whitespace as string delimiter, when splitting, 83 stripping from strings, 79 whitespace in Python code, 2, 12 who function, 57 whos function, 58 who_ls function, 58 Windows servers, managing from Linux, 253– 256 with statement, 101 working directory, installing source in, 264 wrapper( ) function (curses library), 333 wrappers, 9 importing scripts (see import statement) wrapping command-line tools in Python, 397 and changing the behavior, 399 and spawning processes, 401 write( ) method, 101, 104 write( ) method (ftplib module), 156 writelines( ) method, 104 Index | 433



About the Authors Noah Gift has an M.A. in CIS from California State University, Los Angeles, a B.S. in nutritional science from California Polytechnic San Luis Obispo, is an Apple and LPI certified sysadmin, and has worked at companies such as Caltech, Disney Feature Ani- mation, Sony Imageworks, and Turner Studios. In his free time, he enjoys hanging out with his wife, Leah, and their son, Liam, playing the piano, and exercising religiously. Jeremy M. Jones is a software engineer who works for Predictix. His weapon of choice is Python, but he has done some shell, plenty of Perl, a touch of Java, is currently learning C#, and finds functional programming languages (especially OCaml) fascinating. He is the author of the open source projects Munkware, a multiproducer/multicon- sumer, transactional, and persistent queuing mechanism; ediplex, an EDI (electronic data interchange) parsing engine; and podgrabber, a podcast downloader. All three projects were written in the Python language. Jeremy spends his spare time enjoying his family and doing a little writing. He lives in Conyers, Georgia (just east of Atlanta) with his wife, Debra, two children, Zane and Justus, and a lab named Genevieve (how Madelinesque). Opinions and views expressed by Jeremy are his own and not those of Predictix. Colophon The image on the cover of Python for Unix and Linux System Administration is a boa constrictor (boa constrictor). Found throughout South and Central America and some islands in the Caribbean, boa constrictors are non-venomous snakes that can thrive in a wide array of environments, from deserts to open savannas and wet tropical forests, but they prefer arid terrain over wet surroundings. They are both terrestrial and arbor- eal, but as they get older, they tend to spend more time on the ground. Boa constrictors have very unique markings that include diamond- and oval-like pat- terns. Their scales change colors depending on their habitat, allowing them to hide from the forest-dwelling animals that hunt them. In the wild, boa constrictors thrive on small- to medium-size rodents, lizards, bats, birds, mongooses, squirrels, and have even been known to feast on other mammals as large as ocelots. Being cold-blooded and slow moving, boas can go up to a week without eating after capturing large prey. They are solitary and nocturnal hunters, with heat- sensitive pads on their heads to help them hunt. Particularly fond of bats, boas will hang in trees and from the mouths of caves waiting for them to fly by, then they can grab the bats with their mouths. Not surprisingly, boa constrictors kill by constriction. The snake wraps its body around its prey in coils, tightening its grip each time the victim breathes out, eventually suffocating it to death.

Boas are a common attraction in zoos, and they are even relatively common pets. In fact, thousands of dollars are made every year importing them into the U.S. In South America, they are revered as “destroyers of rodents” and are often domesticated for that reason. Boa constrictors grow quite tame in captivity and can live there as such for 20–30 years. Hunted for the exotic pet trade and their decorative markings, some boa constrictors are endangered and have protected status. Boa constrictors are seasonal breeders. To attract males, females emit a scent from their cloacas, which is the chamber into which the intestinal and urogenital tracts discharge. Fertilization happens internally, and females can give birth to up to 60 live babies at one time. Significantly smaller than their anaconda cousins, newborn boas average 2 feet in length and can grow up to 13 feet long and weigh more than 100 pounds. Found in South America, the largest boa constrictor on record was 18 feet! The cover image is a 19th-century engraving from the Dover Pictorial Archive. The cover font is Adobe ITC Garamond. The text font is Linotype Birka; the heading font is Adobe Myriad Condensed; and the code font is LucasFont’s TheSansMonoCondensed.