temp files and tempfile module in python(with examples)

Photo by Agê Barros on Unsplash

First lets understand what a temp or temporary file is.

1. A temporary file is created by a program that serves a temporary purpose and is created due to various reasons such as temporary backup, when a program is manipulating data larger than the systems capability or to break up larger chunks of data into pieces that are easy to process.
2. Temp files have the extension as “.tmp” but they are program dependent(i.e different programs create different temp files).

Most common examples of temp files are

1. Temp Internet Files: This files contain cached info on frequent and recent pages so that these sites load faster.
2. Microsoft Office: For example if an office file is closed abruptly, and when we open that file again we get a message from office to restore previous this data is recovered from the the temporary files associated with the file.

These temp files are updated regularly, but not so often that all work is always saved. Now let's talk about pythons tempfile module.

The tempfile module provides various functionalities to create temporary files.

1. TemporaryFile(): opens and returns unnamed files.
2. NamedTemporaryFile(): opens and returns named file.
3. SpooledTemporaryFile(): Holds files content in memory before writing to disk.
4. TemporaryDirectory: is context manager that removes the directory when the context is closed.

Now let's understand each function in detail.

TemporaryFile

tempfile.TemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None)

1. Returns a file like object that can be used as a temporary storage area. It will be destroyed as soon as it is closed.
2. Under Unix, the directory entry for this file is not created, and other OS do not support this functionality hence This function should be avoided.
3. If suffix is not None the file will end up with that suffix or there will be no suffix. Same applies for the prefix.
4. If the dir is not None the file will be created in the same directory else a default directory is used
5. If suffix,prefix and dir are not None They must be of same type . For example is they are bytes the returned value will also be in bytes.

Where ,When and Why to use this function.

1. Applications that need temporary files to store data, without needing to share that file with other programs.
2. This function creates a file, and on platforms where it is possible, unlinks it immediately this makes impossible for other programs to find or open the file, since there is no reference to the file in the file system table.
3. The file created by TemporaryFile() is removed immediately when it is closed either by calling close() or using with statement.

Examples:

Anonymous temp files are secure because they are not registered in the file system. But if we include dir or prefix or suffix it makes the file less secure but this files can be used for debugging purpose. All the below defined functions take this parameters.

The names are generated using the formula.

import tempfile
with tempfile.TemporaryFile(suffix="_suffix",prefix='prefix_',dir = "") as temp:
    print(temp)
    print(temp.name)

The output follows above formula

NamedTemporaryFile

tempfile.NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None)

1. The function behaviour is similar to TemporaryFile() except that the file is guaranteed to have visible name in the file system.
2. The name can be retrieved from name method of the tempfile object.
3. If delete is true the file is deleted as soon as the it is closed.

Where ,When and Why to use this function.

For applications spanning multiple processes or event hosts, naming the file is simplest way to pass it between applications.

import os
import pathlib
import tempfile
with tempfile.NamedTemporaryFile() as tmp:
    print(tmp)
    print(temp.name)
    f = pathlib.Path(temp.name)
    # the temp file does not exist after closing it.
    f.exists()
Output: 
<tempfile._TemporaryFileWrapper object at 0x0000022C2D7DD808>
C:\Users\sidha\AppData\Local\Temp\tmp8fmo8fed

SpooledTemporaryFile

tempfile.SpooledTemporaryFile(max_size=0, mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None)

1. Operates same like TemporaryFile() except the data is spooled in the memory(data is stored in memory except on the disk) until the file exceeds max_size or the fileno method is called.
2. Once the file exceeds max_size or fileno() is called the contents are written to disk and the operation proceeds with TemporaryFile() .
3. The rollover() method causes the file to rollover to disk regardless of the file size.

return

1. The function returns file like object whose _file attribute is either io.BytesIO or io.TextIOWrapper object depending on whether the file was opened in binary or text mode.(because this function holds the file contents in memory using io.BytesIO or io.TextIOWrapperbuffer until they reach max_size).
2. Or returns a TemporaryFileobject if the rollover() method is called.

Examples:

TemporaryDirectory

tempfile.TemporaryDirectory(suffix=None, prefix=None, dir=None)

1. When several temp files are required we can conveniently use this function to create a temp directory.
2. The returned object of this function can be used as a context manager i.e on completion of the context or destruction of the created object the created temporary directory is and all its contents are removed from the file system.
3. The directory name can be obtained by name attribute of the returned object.
4. If we need to explicitly clean up the file we can use the cleanup() method.

References:

What is a Temporary File? - Definition from Techopedia

tempfile - Generate temporary files and directories - Python 3.8.5 documentation

tempfile - Temporary File System Objects - PyMOTW 3