In actual development, we often meet situations where data needs to be persisted. So-called persistence means moving data from a storage medium that cannot keep data for a long time, usually memory, to a storage medium that can keep data for a long time, usually a hard disk. The most direct and simplest way to make data persistent is to save the data into files through the file system.
A computer file system is a way to store and organize computer data. It makes accessing and finding data easier. The file system uses the abstract logical concepts of files and tree-shaped directories to replace the data block concept of physical devices such as hard disks, optical disks, and flash memory. When users use the file system to save data, they do not need to care which data block on the hard disk the data is actually saved in. They only need to remember the path and file name of the file. Before writing new data, users also do not need to care which data blocks on the hard disk have not been used. The storage-space management of the hard disk, including allocation and release, is automatically completed by the file system. Users only need to remember which file the data was written into.
With a file system, we can read and write data through files very conveniently. In Python, it is very easy to do file operations. We can use Python's built-in open function to open a file. When using the open function, we can specify information such as the file name, operation mode, and character encoding through the function parameters. Then we can read or write the file. The operation mode here means what kind of file to open, text file or binary file, and what kind of operation to do, read, write, or append. The details are shown in the table below.
| Mode | Meaning |
|---|---|
'r' |
Read (default) |
'w' |
Write, truncating existing content first |
'x' |
Write, but raise an exception if the file already exists |
'a' |
Append, writing content to the end of an existing file |
'b' |
Binary mode |
't' |
Text mode (default) |
'+' |
Update mode, allowing both reading and writing |
The figure below shows how to set the operation mode of the open function according to the needs of the program.
When using the open function, if the file opened is a character file, a text file, we can use the encoding parameter to specify the character encoding used to read and write the file. If you do not understand concepts such as character encoding and character set, you can read Character Sets and Character Encodings. We will not repeat that here.
If the open function opens the file successfully, it returns a file object. Through this object, we can read and write the file. If opening the file fails, the open function raises an exception. We will talk about that later. If we want to close the opened file, we can use the close method of the file object. In this way, the file can be released when the file operation ends.
When we use open to open a text file, we need to specify the file name and set the mode to 'r'; if we do not specify it, the default is also 'r'. If we need to specify a character encoding, we can pass the encoding argument. If we do not specify it, the default value is None, which means the operating system's default encoding will be used when reading the file. One thing to keep in mind is that if we cannot guarantee that the encoding used to save the file matches the encoding specified by the encoding argument, reading the file may fail because the characters cannot be decoded.
The example below shows how to read a plain-text file. A plain-text file generally contains only raw encoded characters and, unlike rich text, does not contain formatting control information, so it can be read directly by the simplest text editors.
file = open('致橡树.txt', 'r', encoding='utf-8')
print(file.read())
file.close()Note: To the Oak Tree is a love poem written by Shu Ting in March 1977, and it is also one of my favorite modern poems. The content is shown below:
If I love you I will never be like the climbing trumpet vine, using your high branches to show off myself; If I love you I will never learn from the lovesick bird, repeating a dull song for green shade; Nor only be like a spring, bringing cool comfort all year; Nor only be like a dangerous peak, adding to your height, setting off your dignity. Even sunlight, even spring rain.
No, these are still not enough! I must be a kapok tree beside you, standing together with you as the image of a tree. Roots, tightly holding underground; leaves, touching in the clouds. Every time the wind passes, we greet each other, but no one understands our words. You have your bronze branches and iron trunk, like knives, like swords, also like halberds; I have my red full flowers, like a heavy sigh, also like a brave torch.
We share cold waves, wind, thunder, and lightning; we share mist, flowing clouds, and rainbow colors. Seeming to be always apart, yet depending on each other for the whole life. Only this is great love, and faithfulness is here: Love not only your tall body, but also the place you stand, the land under your feet.
In addition to using the file object's read method, we can also read a text file line by line with a for-in loop, or read all lines into a list with readlines, as shown below.
file = open('致橡树.txt', 'r', encoding='utf-8')
for line in file:
print(line, end='')
file.close()
file = open('致橡树.txt', 'r', encoding='utf-8')
lines = file.readlines()
for line in lines:
print(line, end='')
file.close()If we want to write content into a file, we can open it in mode 'w' or 'a'. The former truncates the existing text and writes new content, while the latter appends new content to the end of what is already there.
file = open('致橡树.txt', 'a', encoding='utf-8')
file.write('\n标题：《致橡树》')
file.write('\n作者：舒婷')
file.write('\n时间：1977年3月')
file.close()Please note that in the code above, if the file specified by open does not exist or cannot be opened, an exception will be raised and the program will crash. To make our code more robust and fault tolerant, we can use Python's exception mechanism to properly handle code that may fail at runtime. Python has five keywords related to exceptions: try, except, else, finally, and raise. Let us first look at the following example and then explain how they are used.
file = None
try:
file = open('致橡树.txt', 'r', encoding='utf-8')
print(file.read())
except FileNotFoundError:
print('无法打开指定的文件!')
except LookupError:
print('指定了未知的编码!')
except UnicodeDecodeError:
print('读取文件时解码错误!')
finally:
if file:
file.close()In Python, we can put code that may have problems at runtime in the try block. After try, we can follow one or more except blocks to catch exceptions and handle them. For example, in the code above, if the file cannot be found, it raises FileNotFoundError; if an unknown encoding is specified, it raises LookupError; and if the file cannot be decoded with the specified encoding when reading, it raises UnicodeDecodeError. So after try we added three except blocks to handle these three different exception situations.
After except, we can also add an else block. This is code that runs when no exception happens in the code in try. Also, the code in else will not do exception catching again. That means if an exception happens there, the program will end because of the exception and report the exception information. Finally, we use the finally block to close the opened file and release the external resource obtained in the program. Because the code in the finally block runs whether the program is normal or has an exception, even if the exit function of the sys module is called to end the Python program, the code in finally still runs. Since the essence of the exit function is raising a SystemExit exception, we call the finally block the "always execute code block". It is most suitable for releasing external resources.
Python has many built-in exception types. In addition to the exception types used in the code above and those we encountered in earlier lessons, there are many others. Their inheritance hierarchy is shown below.
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
+-- StopIteration
+-- StopAsyncIteration
+-- ArithmeticError
| +-- FloatingPointError
| +-- OverflowError
| +-- ZeroDivisionError
+-- AssertionError
+-- AttributeError
+-- BufferError
+-- EOFError
+-- ImportError
| +-- ModuleNotFoundError
+-- LookupError
| +-- IndexError
| +-- KeyError
+-- MemoryError
+-- NameError
| +-- UnboundLocalError
+-- OSError
| +-- BlockingIOError
| +-- ChildProcessError
| +-- ConnectionError
| | +-- BrokenPipeError
| | +-- ConnectionAbortedError
| | +-- ConnectionRefusedError
| | +-- ConnectionResetError
| +-- FileExistsError
| +-- FileNotFoundError
| +-- InterruptedError
| +-- IsADirectoryError
| +-- NotADirectoryError
| +-- PermissionError
| +-- ProcessLookupError
| +-- TimeoutError
+-- ReferenceError
+-- RuntimeError
| +-- NotImplementedError
| +-- RecursionError
+-- SyntaxError
| +-- IndentationError
| +-- TabError
+-- SystemError
+-- TypeError
+-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
+-- ResourceWarning
From the inheritance structure above, we can see that all exceptions in Python are subtypes of BaseException. It has four direct subclasses: SystemExit, KeyboardInterrupt, GeneratorExit, and Exception. Among them, SystemExit indicates that the interpreter is being asked to exit, KeyboardInterrupt indicates that the user interrupted program execution by pressing Ctrl+C, and GeneratorExit indicates that a generator is exiting after an exception notification. It is fine if you do not fully understand these yet. The important one here is Exception, which is the parent type for ordinary exceptions, and many exception classes inherit from it directly or indirectly. If Python's built-in exceptions cannot meet the needs of an application, we can define our own exception types, and custom exceptions should also inherit directly or indirectly from Exception. We can also override or add methods when necessary.
In Python, we can use the raise keyword to raise an exception, and the caller can catch and handle the exception through a try...except... structure. For example, in a function, when the execution condition of the function is not satisfied, we can use raising an exception to tell the caller where the problem is. The caller can recover the code from the exception by catching and handling the exception. The code for defining and raising an exception is shown below.
class InputError(ValueError):
"""Custom exception type"""
pass
def fac(num):
"""Calculate factorial"""
if num < 0:
raise InputError('Factorial can only be calculated for non-negative integers')
if num in (0, 1):
return 1
return num * fac(num - 1)We can call the fac function below, catch the input error with try...except..., and print the exception object. If the input is correct, the program computes the factorial and terminates.
flag = True
while flag:
num = int(input('n = '))
try:
print(f'{num}! = {fac(num)}')
flag = False
except InputError as err:
print(err)For the file object returned by the open function, we can also use with context manager syntax, so that after the file operation is finished, the close method of the file object is automatically executed. This can make the code simpler and cleaner, because we do not need to write the finally block to close the file and release resources. One thing to remind everyone is that not every object can be put in with context syntax. Only objects that match the context manager protocol, that have the magic methods __enter__ and __exit__, can use this syntax. The contextlib module in the Python standard library also provides support for with context syntax. We will explain it later when we use it.
The code rewritten with with is shown below.
try:
with open('致橡树.txt', 'r', encoding='utf-8') as file:
print(file.read())
except FileNotFoundError:
print('无法打开指定的文件!')
except LookupError:
print('指定了未知的编码!')
except UnicodeDecodeError:
print('读取文件时解码错误!')Reading and writing binary files is similar to reading and writing text files, but there are two things to note. First, when using open, the mode is 'rb' for reading and 'wb' for writing. Second, when reading and writing text files, the return value of read and the argument of write are str objects, but when reading and writing binary files, the return value of read and the argument of write are bytes-like objects. The following code copies the image file guido.jpg in the current path into a file named guido-copy.jpg.
try:
with open('guido.jpg', 'rb') as file1:
data = file1.read()
with open('guido-copy.jpg', 'wb') as file2:
file2.write(data)
except FileNotFoundError:
print('Cannot open the specified file.')
except IOError:
print('An error occurred while reading or writing the file.')
print('Program execution finished.')If you want to copy a large binary file, reading all of the data into memory at once may cause excessive memory usage. To reduce memory overhead, we can read and write the file in chunks, as shown below.
try:
with open('guido.jpg', 'rb') as file1, open('guido-copy.jpg', 'wb') as file2:
data = file1.read(512)
while data:
file2.write(data)
data = file1.read()
except FileNotFoundError:
print('Cannot open the specified file.')
except IOError:
print('An error occurred while reading or writing the file.')
print('Program execution finished.')By reading and writing files, we can persist data. In Python, the open function gives us a file object, and then we can use the file object's read and write methods to perform file operations. Problems that occur at runtime can be handled through Python's exception mechanism. The core keywords of Python's exception mechanism are try, except, else, finally, and raise. The except statement after try is not required, and the finally statement is also not required, but at least one of them must exist. There can be one or more except statements. Many except statements are matched in the order they are written. If the exception has already been handled, later except statements will not be entered again. In an except statement, we can also use a tuple to catch many exception types at the same time. If no exception type is written after except, it catches all exceptions by default. After catching an exception, we can use raise to throw it again, but it is not recommended to catch and then throw the same exception again. It is also not recommended to catch all exceptions when you do not clearly understand the logic, because this may hide serious problems in the program. One last point: do not use the exception mechanism to handle normal business logic or control normal program flow. In simple words, do not abuse the exception mechanism. This is a mistake beginners often make.