Introduction
Working with files and directories is a common task when developing in Python. Let's look at several useful tools and methods for working with files and directories.
If you work with files, you might also find my tutorial Working with Binary Data in Python useful.
Working with files
There are several packages that help with working with files in the Python standard library.
For example, os and shutil. There are also several built-in functions. We'll look at
several tasks below.
Does a file exist
The function os.path.exists() lets you check if a path exists. It accepts a string or in PYthon 3.6+ a Path object.
import os
print(os.path.exists('test.txt'))
Get file size
The function os.stat() can be used to get information about a file including its size in bytes. It returns an os.statresult object.
import os
stats = os.stat('test.txt')
print(stats.st_size)
Truncate a file
The functino os.truncate() will let you truncate a file to an arbitrary length:
import os
# Truncate file to be empty
os.truncate('test.txt', 0)
Get file permissions
The function os.stat() can be used to get information about a file including its permissions. It returns an os.statresult.
import os
stats = os.stat('test.txt')
print(stats.st_mode)
print(oct(stats.st_mode)) # Octal output, e.g. 777, 644
Set file permissions (chmod)
To change file permissions, you can use os.chmod().
You can bitwise OR the following options to set the permissions the way you want.
These values come from the stat package: Python stat package documentation.
# import stat
stat.S_IRUSR # Read, user
stat.S_IWUSR # Write, user
stat.S_IXUSR # Execute, user
stat.S_IRGRP # Read, group
stat.S_IWGRP # Write, group
stat.S_IXGRP # Execute, group
stat.S_IROTH # Read, other
stat.S_IWOTH # Write, other
stat.S_IXOTH # Execute, other
stat.S_IRWXU # Read, write, and execute for user
stat.S_IRWXG # Read, write and execute for group
stat.S_IRWXO # Read, write, and execute for other
For example, to set all permissions for user only, you would bitwise OR all the user permissions liek this: S_IRUSR|S_IWUSR|SIXUSR.
This example below shows how to grant all permissions, the equivalent to chmod 777.
import os
import stat
# Equivalent to `chmod 777`
os.chmod('test.txt', stat.S_IRWXU|stat.S_IRWXG|stat.S_IRWXO)
Change ownership (chown)
There are two functions to change ownership: shutil.chown() and os.chown().
The one from shutil is built on top of the os version so we'll focus on that one.
from shutil import chown
# You can use username or uid
# Must provide at least one of user/group
chown('test.txt', user='nanodano')
chown('test.txt', group='sudo')
chown('test.txt', user='root', group='root')
chown('test.txt', user=0, group=0) # root uids
Get file timestamp
The function os.stat() can be used to get information about a file including its timestamps. It returns an os.statresult object.
The times returned are in seconds:
st_atime - last access time st_mtime - last modify time st_ctime - create time (or last metadata change in *nix)
import os
stats = os.stat('test.txt')
print(stats.st_atime)
print(stats.st_mtime)
print(stats.st_ctime)
print(type(stats.st_mtime)) # <class 'float'>
# If you want more readable/usable format use datetime package
import datetime
date_object = datetime.datetime.fromtimestamp(stats.st_ctime)
print(date_object)
print(date_object.strftime('%Y-%m-%d-%H:%M'))
Set file timestamp
The function os.utime() lets you update the timestamp of a file.
import os
import datetime
# To simply update the timestamps to the current time:
os.utime('test.txt')
# To specify the timestamp to set, in seconds as tuple:
# (access_time, modify_time)
time_in_seconds = datetime.datetime.now().timestamp()
os.utime('test.txt', times=(time_in_seconds, time_in_seconds))
Create a file
To create a new file you simply need to open it, and it will automatically create if it does not exist.
To see if a file exists first, use os.exists(). When opening, you will need the w flag at minimum,
to specify it is for writing, otherwise it defaults to reading and will not create a new file.
my_file = open('new_file.txt', 'w')
my_file.close()
Delete a file
The function os.remove() will let you delete a file:
import os
os.remove('test.txt')
Copy a file
You can always copy a file by opening the first one, reading it byte-by-byte, and writing the contents to a new file, but there is a more convenient way with shutil.copy().
import shutil
shutil.copy('test.txt', 'test_duplicate.txt')
Move a file
You have a couple options for moving/renaming a file:
Let's look at shutil.move():
import shutil
shutil.move('test.txt', 'data.txt')
Get file extension
Use os.path.splitext()
to split a file path apart and grab the file extension. The return value is a tuple that
includes the filename and the extension separated. The file extension includes the ..
import os
print(os.path.splitext('test.txt'))
# Output: ('test', '.txt')
Write to a text file
To write a file, open it with w mode. You can use the with statement to automatically
handle closing the file. Use write() to output data to the file.
with open('test.txt', 'w') as my_file:
my_file.write('Hello, world!\n')
Append to a binary file
To open a file for append, include the a flag instead of w, but it will still
create the file if it does not exist.
To open a file in binary mode instead of text, include the b flag.
Use write() just like you would with text, but you can include raw bytes.
with open('data.dat', 'ab') as my_binary_file:
my_binary_file.write(b'\x00\x00\xFF\xFF')
Read entire file contents in to memory
To read the entire file contents in to memory at once, you can simply call read() with
no parameters. This applies to text and binary files.
Here is an example with a text file where you get a str object:
with open('test.txt') as my_file:
data = my_file.read()
print(type(data)) # <class 'str'>
print(len(data))
print(data)
Here is an example with a binary file where you get a bytes object:
with open('test.txt', 'rb') as my_binary_file:
data = my_binary_file.read()
print(type(data)) # <class 'bytes'>
print(len(data))
print(data)
Read all lines from a text file
To open a text file for reading, you don't need to provide any flags when opening since those are the default settings.
File objects that you have open for reading have a readlines() to easily
with open('test.txt') as my_file:
lines = my_file.readlines()
print(len(lines))
print(type(lines)) # <class 'list'>
for line in lines:
print(line.strip())
Read specific number of bytes
To read a specific number of bytes from a file, you just call read() with a parameter
that specifies how many bytes to read. After you read, it advances the file cursor forward
that many positions as well, so the next read will continue moving forward in the file.
with open('test.txt', 'rb') as my_binary_file:
one_byte = my_binary_file.read(1) # Read only a single byte
print(one_byte)
Seek a position in a file
You may want to jump to the beginning of a file, somewhere in the middle, or directly to the end.
With seek() you can jump anywhere you want. If you provide only a single argument, it will take
you directly to that byte position. If you specify a second argument, you can tell seek to start
relative to the beginning, your current cursor position, or the end of the file.
Options:
os.SEEK_SET - Set position from beginning
os.SEEK_CUR - Seek relative to current cursor position
os.SEEK_END - Seek relative to end of file
Example usage:
import os
with open('test.txt', 'rb') as my_file:
# Jump to beginning of file
my_file.seek(0)
# Equivalent, jump to beginning
my_file.seek(0, os.SEEK_SET)
# Read 2 bytes, moving cursor forward
print(my_file.read(2))
# Move 2 bytes backwards from current cursor (back to beginning)
my_file.seek(-2, os.SEEK_CUR)
# Re-read the same two bytes
print(my_file.read(2))
# Go to the very end of the file
my_file.seek(0, os.SEEK_END)
# Move back two bytes from current position (end of file)
my_file.seek(-2, os.SEEK_CUR)
# Read last two bytes of the file
print(my_file.read(2))
Is a file a symlink?
Use os.path.islink() to find out if a file
is a symbolic link. This will always fetch latest information, where os.is_symlink() may fetch cached results.
import os
print(os.path.islink('test.txt'))
Create a symlink
Use os.symlink() to create a symbolic link.
import os
os.symlink('test.txt', 'symlink_to_test.txt')
Check hard file links
The function os.stat() can be used to get information about a file including info about its hard links. It returns an os.statresult.
import os
stats = os.stat('test.txt')
# Number of hard links (Should always be at least 1, itself)
print(stats.st_nlink)
Create a hard file link
To create a hard link, use os.link():
import os
os.link('test.txt', 'hard_linked_test.txt')
Start a file with default application
The function os.startfile() can be used (only in Windows) to launch a file using its default application. It is like double clicking the file from the Windows Explorer.
import os
os.startfile('text.txt')
Get absolute path of a file
If you only have a filename but you need to full absolute path of the file, you can use os.path.abspath.
import os
print(os.path.abspath('test.txt'))
Get name of current running Python script
The dunder __file__ contains the name of the current running Python script. By itself though, it is only the file name and does not include the full absolute path. To get that, you want to use os.path.abspath().
import os
print(__file__)
print(os.path.abspath(__file__))
Get directory name for a file
Use os.path.dirname to get get just the directory path to a file without including the file itself.
import os
# Get the directory name where the current running Python file resides
print(os.path.dirname(os.path.abspath(__file__)))
Working with directories
While files and directories have a lot in common, and are ultimately both treated as files, there are some operations you want to perform on a directory that you don't necessarily want to do on files. Let's look at some common tasks with directories.
Print current working directory
Use os.getcwd() to get the directory
you are currently in, just like pwd.
import os
print(os.getcwd())
Change working directory
Use os.chdir() to change directories just like using cd.
You can use .. and relative or absolute paths.
import os
os.chdir('/')
Join directory paths
You can use os.path.join() to join directories using the system-specific directory separator. This is better than manually combining directories with plus signs and slashes yourself.
import os
# Generate a relative path using system-specific directory separators
print(os.path.join('path', 'to', 'myfile.txt'))
Normalize a path
If you have an ugly path that has redundant slashes or includes relative operators, you can clean it up using the os.path.normpath() function.
import os
ugly_path = '/home//nanodano/../nanodano//test.txt'
pretty_path = os.path.normpath(ugly_path)
print(pretty_path) # /home/nanodano/test.txt
Make directories
There are a few options for creating directories:
Let's look at each one:
import os
# Make a single directory
os.mkdir('new_directory')
import os
# Create a directory including parent directories
os.makedirs('path/to/new/dirs')
from pathlib import Path
# Create a directory including parent directories
target_path = Path('/path/to/new/dirs')
target_path.mkdir(parents=True, exist_ok=True)
Is a directory a mount point?
Use os.path.ismount() to see
if a directory is a mount point. For example, in Windows Subsystem for Linux (WSL), /mnt/c is
the default mount point to your Windows drive.
import os
os.path.ismount('/mnt/c')
List contents of a directory
Use os.listdir() to get the contents
of a directory, excluding . and ..:
import os
dir_contents = os.listdir(os.getcwd())
print(type(dir_contents)) # <class 'list'>
print(dir_contents)
for entry in dir_contents:
print(entry)
Check if path is file or directory
Building on the previous example to list directory contents, we can inspect the entry to see if it is a file or a directory. You can use os.path.isdir() and os.path.isfile() to check if they are a file or directory.
import os
print(os.path.isdir('test.txt'))
print(os.path.isfile('test.txt'))
Walk a directory
You can use os.walk() to recursively go through directories.
Here is a very basic example of how to use it:
import os
# Recursively go through all directories
for (root_path, directories, files) in os.walk(os.getcwd()):
print(root_path)
print(directories)
print(files)
For more detailed examples, see my tutorial Walk a Directory in Python.
Copy directory tree
Use shutil.copytree() to recursively copy a directory and its contents.
import shutil
shutil.copytree('original_dir', 'copied_dir')
# In 3.8, the `dirs_exist_ok` option is available
# shutil.copytree('original_dir', 'copied_dir', dirs_exist_ok=True)
Delete a directory
Use os.rmdir() to remove a single directory.
import os
# Remove a single directory
os.rmdir('my_useless_dir')
Delete directory tree
To remove an entire directory tree, including sub-directories, you can use shutil.rmtree().
import shutil
# Recursively delete the directory and anything inside
shutil.rmtree('my_useless_dir')
Get user home directory
The function os.path.expanduser()
will let you use the special character ~ to represent the user's home directory.
import os
print(os.path.expanduser('~/Downloads'))
Conclusion
After reading this guide you should have a good understanding of how to do many common tasks with files and directories in Python.