14.3: Files in Different Locations and Working with CSV Files

Learning Objectives

By the end of this section you should be able to

• Demonstrate how to access files within a file system.
• Demonstrate how to process a CSV file.

Opening a file at any location

When only the filename is used as the argument to the open() function, the file must be in the same folder as the Python file that is executing. Ex: For fileobj = open("file1.txt") in files.py to execute successfully, the file1.txt file should be in the same folder as files.py.

Often a programmer needs to open files from folders other than the one in which the Python file exists. A path uniquely identifies a folder location on a computer. The path can be used along with the filename to open a file in any folder location. Ex: To open a file named logfile.log located in /users/turtle/desktop the following can be used:

fileobj = open("/users/turtle/desktop/logfile.log")

Table 14.2 Opening files on different paths. In each of the following cases, a file called output.txt is located in a different folder than the Python folder. Windows uses backslash \ characters instead of forward slash / characters for the path. If the backslash is included directly in open() , then an additional backslash is needed for Python to understand the location correctly.

Operating System	File location	open() function example
Mac	/users/student/	fileobj = open("/users/student/output.txt")
Linux	/usr/code/	fileobj = open("/usr/code/output.txt")
Windows	c:\projects\code\	fileobj = open("c:/projects/code/output.txt") or fileobj = open("c:\\projects\\code\\output.txt")

Concepts in Practice: Opening files at different locations

For each question, assume that the Python file executing the open() function is not in the same folder as the out.txt file.

Each question indicates the location of out.txt, the type of computer, and the desired mode for opening the file. Choose which option is best for opening out.txt.

Concepts in Practice \(\PageIndex{1}\)

/users/turtle/files on a Mac for reading

1. fileobj = open("out.txt")
2. fileobj = open("/users/turtle/files/out.txt")
3. fileobj = open("/users/turtle/files/out.txt", 'w')

Answer

b. The path followed by the filename enables the file to be opened correctly and in read mode by default.

Concepts in Practice \(\PageIndex{2}\)

c:\documents\ on a Windows computer for reading

1. fileobj = open("out.txt")
2. fileobj = open("c:/documents/out.txt", 'a')
3. fileobj = open("c:/documents/out.txt")

Answer

c. The use of forward slashes / replacing the Windows standard backslashes \ enables the path to be read correctly. The path followed by the filename enables the file to be opened and when reading a file read mode is preferable.

Concepts in Practice \(\PageIndex{3}\)

/users/turtle/logs on a Linux computer for writing

1. fileobj = open("out.txt")
2. fileobj = open("/users/turtle/logs/out.txt")
3. fileobj = open("/users/turtle/logs/out.txt", 'w')

Answer

c. The file is created or overwritten and changes can be written into the file.

Concepts in Practice \(\PageIndex{4}\)

c:\proj\assets on a Windows computer in append mode

1. fileobj = open("c:\\proj\\assets\\out.txt", 'a')
2. fileobj = open("c:\proj\assets\out.txt", 'a')

Answer

a. Since the backslashes in the Windows path appear in the string argument, which usually tells Python that this is part of an escape sequence, the backslashes must be ignored using an additional backslash \ character.

Working with CSV files

In Python, files are read from and written to as Unicode by default. Many common file formats use Unicode such as text files (.txt), Python code files (.py), and other code files (.c,.java).

Comma separated value (CSV, .csv) files are often used for storing tabular data. These files store cells of information as Unicode separated by commas. CSV files can be read using methods learned thus far, as seen in the example below.

Figure 14.2 CSV files. A CSV file is simply a text file with rows separated by newline \n characters and cells separated by commas.

Raw text of the file:

Title, Author, Pages\n1984, George Orwell, 268\nJane Eyre, Charlotte Bronte, 532\nWalden, Henry David Thoreau, 156\nMoby Dick, Herman Melville, 538

Example 14.3: Processing a CSV file

    """Processing a CSV file."""
    # Open the CSV file for reading
    file_obj = open("books.csv")

    # Rows are separated by newline \n characters, so readlines() can be used to read in all rows into a string list
    csv_rows = file_obj.readlines()

    list_csv = []

    # Remove \n characters from each row and split by comma and save into a 2D structure
    for row in csv_rows:
      # Remove \n character
      row = row.strip("\n")
      # Split using commas
      cells = row.split(",")
      list_csv.append(cells)
    
    # Print result
    print(list_csv)

The code's output is:

    [['Title', ' Author', ' Pages'], ['1984', ' George Orwell', ' 268'], ['Jane Eyre', ' Charlotte Bronte', ' 532'], ['Walden', ' Henry David Thoreau', ' 156'], ['Moby Dick', ' Herman Melville', ' 538']]

Concepts in Practice \(\PageIndex{5}\): File types and CSV files

Why does readlines() work for reading the rows in a CSV file?

1. readlines() reads line by line using the newline \n character.
2. readlines() is not appropriate for reading a CSV file.
3. readlines() automatically recognizes a CSV file and works accordingly.

Answer

a. The newline \n character indicates where line breaks are in a file and is used by the readlines() function to tell lines apart.

Concepts in Practice \(\PageIndex{6}\): File types and CSV files

For the code in the example, what would be the output for the statement print(list_csv[1][2])?

1. 532
2. 268
3. Jane Eyre

Answer

b. The cell at the 2nd row and 3rd column contains '268' .

Concepts in Practice \(\PageIndex{7}\): File types and CSV files

What is the output of the following code for the books.csv seen above?

file_obj = open("books.csv")
csv_read = file_obj.readline()
print(csv_read)

1. ['Title, Author, Pages\n', '1984, George Orwell, 268\n', 'Jane Eyre, Charlotte Bronte, 532\n', 'Walden, Henry David Thoreau, 156\n', 'Moby Dick, Herman Melville, 538']
2. [['Title', ' Author', ' Pages'], ['1984', ' George Orwell', ' 268'], ['Jane Eyre', ' Charlotte Bronte', ' 532'], ['Walden', ' Henry David Thoreau', ' 156'], ['Moby Dick', ' Herman Melville', ' 538']]
3. Title, Author, Pages

Answer

c. The first line of the books.csv is read into csv_read. The first line is 'Title, Author, Pages' , which is the same as the first row.

Exploring further

Files such as Word documents (.docx) and PDF documents (.pdf), image formats such as Portable Network Graphics (PNG, .png) and Joint Photographic Experts Group (JPEG, .jpeg or .jpg) as well as many other file types are encoded differently.

Some types of non-Unicode files can be read using specialized libraries that support the reading and writing of different file types.

• PyPDF is a popular library that can be used to extract information from PDF files.
• BeautifulSoup can be used to extract information from XML and HTML files. XML and HTML files usually contain unicode with structure provided through the use of angled <> bracket tags.
• python-docx can be used to read and write DOCX files.
• Additionally, csv is a built-in library that can be used to extract information from CSV files.

Try It: Processing a CSV file

The file fe.csv contains scores for a group of students on a final exam. Write a program to display the average score.

Interactive Code

fe.csv

student id, score
123, 95
213, 92
111, 86
555, 97
621, 99
777, 100
312, 84
391, 88
398, 87
444, 95

Answer

# Open the CSV file for reading
file_obj = open("fe.csv")

# Rows are separated by newline \n characters, so readlines() can be used to read in all rows into a string list
csv_rows = file_obj.readlines()

list_csv = []

summation = 0.0

for n in range(1, len(csv_rows)):
# Split using commas
cells = csv_rows[n].split(",")
summation += float(cells[1])


# Print result
print("Average score: ", summation/(len(csv_rows)-1))n
# Close the file
nfile_obj.close()