B2.5 File Processing

B2.5.1 Construct code to perform file processing operations. Examples in Python.

Why Files

Lists and variables live in RAM and are lost when the program ends. Files are saved in secondary storage, so the data persists after the program closes.

Sequential Files

A sequential file stores records one after another and can only be read in order, from the start. You cannot skip to a record.

Advantages	Disadvantages
Simple, low overhead	Slow to find one record: must read from the start (linear search)
Good when every record is processed in order	Changing or inserting in the middle means rewriting the file

Uses: system log files, batch processing (e.g. a bank processing transactions overnight), archives of old data.

File Modes

Mode	Does	If file doesn't exist
`"r"` read	Reads the file (default)	FileNotFoundError
`"w"` write	Overwrites everything already in the file	Creates it
`"a"` append	Adds to the end, keeps old data	Creates it
`"r+"` read and write	Reads and writes the existing file	FileNotFoundError

with open(...) as f: closes the file automatically at the end of the block, even after an error. No f.close() needed.

Write and Append

names = ["Ana", "Cate", "Jamie"]

with open("names.txt", "w") as f:      # creates or OVERWRITES
    for name in names:
        f.write(name + "\n")           # write() adds no newline itself

with open("names.txt", "a") as f:      # adds to the END
    f.write("Ian\n")

with open("names.txt", "r") as f:
    print(f.read())

Ana
Cate
Jamie
Ian

write() and writelines() don't add line breaks. Add "\n" yourself, or every record ends up on one line.

Read

Method	Returns
`f.read()`	the whole file as one string
`f.readline()`	the next single line
`f.readlines()`	a list of lines (each still ending in `\n`)
`for line in f:`	each line in turn

with open("names.txt", "r") as f:
    for line in f:                     # one line at a time, in order
        print(line.strip())            # strip() removes the "\n"

Ana
Cate
Jamie
Ian

with open("names.txt", "r") as f:
    names = f.read().splitlines()      # whole file into a list
print(names)
print(len(names), "records")

['Ana', 'Cate', 'Jamie', 'Ian']
4 records

Searching a sequential file

target = "Jamie"
found = False
with open("names.txt", "r") as f:
    for line in f:                     # must start at record 1
        if line.strip() == target:
            found = True
            break                      # stop once found
print(target, "found:", found)

Jamie found: True

Missing file

try:
    with open("missing.txt", "r") as f:
        print(f.read())
except FileNotFoundError:
    print("File not found")

File not found

Practice

scores.txt contains 3 lines. A program opens it with "w" and writes 2 lines. How many lines are in the file now? What if it used "a"?

"w": 2 lines (old content overwritten). "a": 5 lines (added to the end).

Advantage: simple to create with low overhead [1], efficient when every record is processed in order, such as batch processing [1].
Disadvantage: records can only be accessed in order from the start [1], so finding one record in a large file is slow [1].

Read a text file, count how many times a word appears, and suggest the mood if it appears often.

text = "We celebrate today. Fans celebrate the win and celebrate again."
with open("article.txt", "w") as f:
    f.write(text)

with open("article.txt", "r") as f:
    words = f.read().lower().split()

count = 0
for w in words:
    if w.strip(".,!?") == "celebrate":
        count = count + 1

print("celebrate appears", count, "times")
if count > 2:
    print("Mood: positive")

celebrate appears 3 times
Mood: positive

Ninja Notes:

Files persist after the program ends. Variables don't.
Sequential file = records in order, read from the start, no skipping.
"w" overwrites, "a" appends, "r" reads.
Use with open() so the file always closes.
Add "\n" when writing, .strip() when reading.
Catch FileNotFoundError when reading.