Using osfclient to download project files



If downloading your files from the OSF website using Download as ZIP does not work, you can use a tool called osfclient instead.

This can be especially useful for:

  • Very large projects;
  • projects containing many files or folders;
  • ZIP files that download but cannot be opened or unpacked.

You do not need to know Python programming to use osfclient. You will need to install Python and enter several commands in Terminal on a Mac or PowerShell on Windows. Osfclient will download files only. To download metadata, wikis, and activity logs from projects, refer to this guide.


Before you begin, you will need:

  • access to the OSF project you want to download;
  • enough free space on your computer or an external drive;
  • a stable internet connection;
  • the OSF project ID;
  • Python installed on your computer (described below);
  • an OSF Personal Access Token if the project is private (described below).

Important: projects with components

An OSF project and its components are separate.

Downloading the main project with osfclient does not automatically download the files in its components.

If your project contains components, repeat the download process for:

  1. the main project; and
  2. each component that contains files you need.

Each component has its own OSF ID or GUID


Step 1: Find your OSF project GUID

The GUID is the five-character identifier in the project URL.

For example in this project:

https://osf.io/svqht/overview

The project GUID is the short group of letters and numbers:


svqht


A component works the same way.


Keep this GUID handy.


Step 2: Create a personal access token to access private projects


1. Sign in to the OSF.

2. On the left-hand side of the screen, click “Settings”, then select “Personal Access Tokens”.

3. Select “Create Token”

4. Enter a name for the token, such as `osfclient`. If you already have a token with 'osf.full_read' scopes, you can use that instead.

5. Select the `osf.full_read` scope. Write access is not required.


6. Create the token, then copy it immediately. The OSF displays the token only once.

Treat the token like a password. Do not place it in a script, include it in a screenshot, or share it with anyone. You can delete the token from your OSF settings when the download is complete if you wish.


For more information, see Profile and Account: Create a Personal Access Token.


Step 3:  Install or check Python

Open Terminal or Powershell.

You can find Terminal (Mac) or Powershell (Windows) by:


On macOS, open Terminal by

  1. Opening Applications.
  2. Opening Utilities.
  3. Clicking Terminal.

and run (type the following text and hit enter):

python3 --version


If the command displays Python 3 followed by a version number (e.g. if it says something like, “Python 3.9.6”), continue to the next section.

Python 3.14.6



Python is installed.

If you receive an error saying the command cannot be found, install Python 3 from the Python website: https://www.python.org/downloads/

If Python is not installed, download Python 3 from  Python website.  After Python is installed, attempt Step 2 again.


On Windows, open PowerShell by

  1. Click the Start button or press the Windows key.
  2. Type PowerShell.
  3. Click Windows PowerShell or PowerShell in the search results.

and run (type the following text):

python --version

Press Return.


If that does not work, try:

py --version

If you see a Python 3 version number, continue to the next step.

Otherwise, install Python 3 from the Python website. After Python is installed, attempt Step 2 again.


Step 4: Install osfclient

You only need to do this once.


On Mac


In Terminal, enter:


python3 -m pip install osfclient

On Windows

In PowerShell, enter:

python -m pip install osfclient

If that does not work, try:

py -m pip install osfclient

Test the Installation:

On Mac and On Windows

When installation finishes, test osfclient by entering:


osf -h

You should see a list of osfclient commands:

On Mac

You should see a list of osfclient commands.

If you see:

command not found: osf

close Terminal or PowerShell, reopen it, and try again.


Step 5: Give osfclient your token


The following command temporarily makes the token available to osfclient.

on Mac

In Terminal, enter:

export OSF_TOKEN="PASTE_YOUR_TOKEN_HERE"

Replace PASTE_YOUR_TOKEN_HERE with the token you created in Step 2.


For example, the command should have this general form:


export OSF_TOKEN="your-token-goes-here"

There should be:


  • no colon before OSF_TOKEN
  • no spaces around the = sign
  • quotation marks around the token


On Windows

In PowerShell, enter:

$env:OSF_TOKEN="PASTE_YOUR_TOKEN_HERE"

Replace PASTE_YOUR_TOKEN_HERE with the token you created in Step 2.

For example:

$env:OSF_TOKEN="your-token-goes-here"

The token will normally remain available until you close that Terminal or PowerShell window.

Once you have entered your token, do not close or quit Terminal or PowerShell or you will likely need to reenter your token to have the appropriate permissions.


Step 6: Test access before downloading


Before starting a potentially very large download, make sure osfclient can see the project. These commands are the same for Mac (Terminal) and Windows (PowerShell).


On Mac

In Terminal, enter:

osf -p PROJECTID ls

Replace PROJECTID with your GUID from Step 1.


For example:

osf -p svqht ls

You should see file paths from the project.

On Windows

In PowerShell, enter:

osf -p PROJECTID ls

Replace PROJECTID with your GUID from Step 1.


For example:

osf -p svqht ls

If you see the files you expect, continue.

For example:

TRUST PDF.pdf
Full_dataset_of_pappers_formatted.xls
enter token step video.mov
Social Cognitions and Social Frailty Project _ Method and Measures.pdf
Social Cognitions and Social Frailty Project _ Overview Research Question Hypotheses.pdf
Social Cognitions and Social Frailty Project _ Analytic Approach.pdf
Social Cognitions and Social Frailty Project _ References.pdf
Agent Laboratory.pdf

Step 7: Choose where the files should be downloaded

There is no Terminal/PowerShell entry on this step, only determining the download location that you will utilize in the following steps.


You can download the files to:

  • your Desktop
  • another folder on your computer
  • an external hard drive

For large projects, make sure the destination has enough free space.

We recommend creating a separate folder for each OSF project or component. Using the OSF GUID as the folder name makes it easier to keep multiple downloads organized.

You do not need to create this folder first. osfclient   will create the necessary folders as files are downloaded based on your download location preferences.


On Mac

In Terminal, a convenient location is:

$HOME/Desktop/OSF-downloads/svqht

On Windows

In PowerShell, a convenient location is:

$HOME\Desktop\OSF-downloads\svqht

If your Windows Desktop is managed through OneDrive, your Desktop may be in a different location. You can find the correct Desktop path by entering:

[Environment]::GetFolderPath("Desktop")

Download to an external drive

On Mac

External drives normally appear under:

/Volumes/

To see the names of connected drives, enter:

ls /Volumes

For example, if your external drive is named My External Drive          , you could use:

/Volumes/My External Drive/OSF-downloads/svqht

Keep quotation marks around paths containing spaces when you use them in a command.


On Windows

External drives usually have a drive letter, such as E:          .

For example:

E:\OSF-downloads\svqht

Step 8: Download the project

In Terminal, the basic command is

osf -p PROJECTID clone -U "DOWNLOAD_LOCATION"

In PowerShell, the basic command is

osf -p PROJECTID clone -U "DOWNLOAD_LOCATION"

The -U option is important for large downloads. It allows you to run the command again after an interruption without unnecessarily downloading files that already match the files on OSF.


On Mac Download to your desktop

For example, to download to desktop:

osf -p svqht clone -U "$HOME/Desktop/OSF-downloads"

Press Return or Enter.


The download will begin.


For a large project, this may take several hours.


The downloaded folder will contain additional folders for the storage providers used by the OSF project, such as osfstorage .

On Mac Download to an external drive

osf -p svqht clone -U "/Volumes/My External Drive/OSF-downloads/svqht"

Press Return or Enter.


The download will begin.


For a large project, this may take several hours.


The downloaded folder will contain additional folders for the storage providers used by the OSF project, such as osfstorage .


On Windows Download to your desktop

osf -p svqht clone -U "$HOME\Desktop\OSF-downloads\svqht"

Press Return or Enter.


The download will begin.


For a large project, this may take several hours.


The downloaded folder will contain additional folders for the storage providers used by the OSF project, such as osfstorage .


On Windows Download to an external drive

osf -p svqht clone -U "$HOME\Desktop\OSF-downloads\svqht

Press Return or Enter.


The download will begin.


For a large project, this may take several hours.


The downloaded folder will contain additional folders for the storage providers used by the OSF project, such as osfstorage          .

Do not close the Terminal or PowerShell window while the download is running.

The files will download to the referenced folder.

Step 9: Keep a Mac awake during a long download


If you are using a Mac, you can prevent it from going to sleep during the download.


Open a second Terminal window and enter:

caffeinate -dimsu

Leave this second Terminal window open.


When the OSF download is complete, return to the window running caffeinate and press:


Control + C


Step 10: If the download stops


Do not immediately delete the files that were downloaded.


In many cases, you can simply run the same command again.


For example:

osf -p svqht clone -U "/Volumes/My External Drive/OSF-downloads/svqht"

Because you used -U, osfclient checks files that already exist locally. Files that already match the OSF copy can be skipped, allowing the download to continue with files that are missing or different.


Step 11: Download components separately


If your project contains components, repeat the process for each component.


For example, suppose you have:


Main project: abc12

Data component: def34

Analysis component: ghi56


You could download them into separate folders (on Desktop):

osf -p abc12 clone -U "$HOME/Desktop/OSF-downloads/abc12"
osf -p def34 clone -U "$HOME/Desktop/OSF-downloads/def34"
osf -p ghi56 clone -U "$HOME/Desktop/OSF-downloads/ghi56"

For an external drive:


osf -p abc12 clone -U "/Volumes/My External Drive/OSF-downloads/abc12"
osf -p def34 clone -U "/Volumes/My External Drive/OSF-downloads/def34"
osf -p ghi56 clone -U "/Volumes/My External Drive/OSF-downloads/ghi56"

Repeat this for every component containing files you want to preserve.


Step 12: Check your download


After the command finishes:


  • Open the downloaded folder in Finder or File Explorer
  • Confirm that expected folders and files are present
  • Open several files to make sure they work
  • Check especially large or important files
  • Repeat the process for any components

Do not delete the OSF copy until you are satisfied that your downloaded copy is complete.


For important research data, consider keeping three copies in separate locations.



Troubleshooting


zsh: command not found: osf

osfclient is either not installed or Terminal cannot find it.


Try:

python3 -m pip install osfclient

Then close Terminal, reopen it, and run:

osf -h
Please set a username

If you are downloading a private project and are using a Personal Access Token, first make sure the token is actually set (Step 5).


On Mac:

printenv OSF_TOKEN | wc -c


Windows PowerShell:

$env:OSF_TOKEN.Length


The result should be greater than zero. If not, set the token again:


On Mac:

export OSF_TOKEN="PASTE_YOUR_TOKEN_HERE"


Windows PowerShell:

$env:OSF_TOKEN="PASTE_YOUR_TOKEN_HERE"


Then retry:

osf -p PROJECTID ls
You are not authorized to access this project

Check that:

You are signed in to the OSF account that can access the project;

The project ID is correct;

Your Personal Access Token was copied correctly;

The token has osf.full_read permission;

The token has not been revoked.


Creating a new Personal Access Token and trying again may resolve the problem.
Response has status code 500

A message similar to:

RuntimeError: Response has status code 500


means OSF returned a server error while osfclient was retrieving a file. Try running the same download command again:

osf -p PROJECTID clone -U "DOWNLOAD_LOCATION"


If one particular file continues to fail, try downloading that file directly from the OSF UI.
No space left on device

The destination does not have enough available storage.


Choose another location or an external hard drive with sufficient space.
Permission denied when using an external drive on Mac

First make sure you are using the actual name of the mounted drive.

In Terminal, run:

ls /Volumes


Then use the exact name shown there. For example:

osf -p abc12 clone -U "/Volumes/Extreme SSD/OSF-downloads/abc12"


Unless that is actually the name of your drive, do not use a placeholder such as:

/Volumes/My External Drive/

After the download


When you are completely finished, you can remove the token from the current Terminal session.


Mac

unset OSF_TOKEN

Windows PowerShell

Remove-Item Env:OSF_TOKEN

If you created the token only for this download, you can also return to OSF Account Settings → Personal Access Tokens and delete it.


Back to Main Support page

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.