Google Drive File Stream Raspberry Pi

I see my Drive FOLDER listed there, but not the Drive drive. When I manually navigate to the Desktop, I see only the drive and no folder, but the drive has a file icon next to it and it's entirely greyed out. Any help is thoroughly appreciated. The Pi Discord has been an amazing help, but I can't seem to figure out what the problem would. The “drive push” command pushes a file to Google Drive, and the “drive pull” command pulls a file from Google Drive. The developer notes situations where this could be especially useful—if you store a virtual machine in your Google Drive, you might want to immediately sync a tiny text file rather than syncing the large virtual machine.

18-NOV-2016

If you are running RetroPie on Raspberry Pi to play retro games you have probably spent more time setting it up and configuring it than actually playing. And if you have two or more...

Wouldn't it be nice to manage the roms folder on a Google Drive folder and let it sync to one or more RetroPies? And if you spent time scraping metadata and box art on one RetroPie that would sync to the others without additional work.

It's kind of possible. But it is slow and you will have to initialize the syncing and choose sync direction manually.

Please note: this might not be the best way to do it. It might not even be a good way to do it. In fact, this might be a horrible way to do it and you may loose all your data and lots of sleep so do it at your own risk and make backups of everything.

You'll need the semi-official Google Drive application for Raspian and you'll find it here:

First, log in to your RetroPie using SSH and create a directory for Google drive and inside that the folder we wish to sync. I created a folder called 'drive' and inside that 'RetroSync'. We want to keep everything here inside one folder and not sync the entire Google Drive.

Then you need to check what version of the drive binary you need by running:

In my case, running a Raspberry Pi 3, it's ARM v7 - and it will tell you so four times, probably because there are four cores in the CPU.

You can find the URL's to the different version at:

https://github.com/odeke-em/drive/releases/

Copy the right URL and download it with wget. In my case it looks like this:

That will download the binary file into an application called 'drive' located in /usr/local/bin.

Before you can run it though, you need to tell the operating system it is an executable binary:

Once you'we done that, you can run the command and the first time you need to initialize it by telling drive where your sync folder is (the one we created previously):

When you initialize drive, it will give you a rather long URL beginning with accounts.google.com. Copy the entire URL and paste it into a browser (typically on another computer) where you are logged in as the Google user who's account you wish to use. You will be asked if you wish to grant the application 'drive' access to your Google Drive - and if you select yes, you get a key string that you need to copy and paste back into your Raspberry Pi (it will be asking for the verification code).

Now Google Drive is ready to be used on your Raspberry Pi.

But now you have to move the stuff you want to sync into the sync folder. Instead of actually moving files and messing up RetroPie, we create symbolic links in our RetroSync folder to the folders with the content we wish to sync:

The most important folder is the RetroPie folder containing roms and bios files so let's make a link to that:

But this only syncs the roms. If you have spent a lot of time getting metadata and boxart, that has to be synced as well and that stuff is in two other directories (box art and metadata such as descriptions).


And now we are ready to sync.

You'll need to move into the sync directory to sync:

and then you will need to either upload (push) your stuff to Google Drive if you are using a RetroPie with data or download (pull) if you are setting up a new empty RetroPie by running

or

The application 'drive' will analyze the environment, list what files are going to go where and ask for confirmation (upper case 'Y' or lower case 'n').

Then you just have to wait a few hours. The first sync is slow but after that only changes are copied and that is much faster.

Google Drive

Raspberry Pi Google Voice

Paths are specified as drive:path

Drive paths may be as deep as required, e.g. drive:directory/subdirectory.

The initial setup for drive involves getting a token from Google drivewhich you need to do in your browser. rclone config walks youthrough it.

Here is an example of how to make a remote called remote. First run:

This will guide you through an interactive setup process:

Note that rclone runs a webserver on your local machine to collect thetoken as returned from Google if you use auto config mode. This onlyruns from the moment it opens your browser to the moment you get backthe verification code. This is on http://127.0.0.1:53682/ and thisit may require you to unblock it temporarily if you are running a hostfirewall, or use manual mode.

You can then use it like this,

List directories in top level of your drive

List all the files in your drive

To copy a local directory to a drive directory called backup

Scopes

Rclone allows you to select which scope you would like for rclone touse. This changes what type of token is granted to rclone. Thescopes are definedhere.

The scope are

drive

This is the default scope and allows full access to all files, exceptfor the Application Data Folder (see below).

Choose this one if you aren't sure.

drive.readonly

This allows read only access to all files. Files may be listed anddownloaded but not uploaded, renamed or deleted.

drive.file

With this scope rclone can read/view/modify only those files andfolders it creates.

So if you uploaded files to drive via the web interface (or any othermeans) they will not be visible to rclone.

This can be useful if you are using rclone to backup data and you wantto be sure confidential data on your drive is not visible to rclone.

Files created with this scope are visible in the web interface.

drive.appfolder

This gives rclone its own private area to store files. Rclone willnot be able to see any other files on your drive and you won't be ableto see rclone's files from the web interface either.

drive.metadata.readonly

This allows read only access to file names only. It does not allowrclone to download or upload data, or rename or delete files ordirectories.

Root folder ID

You can set the root_folder_id for rclone. This is the directory(identified by its Folder ID) that rclone considers to be the rootof your drive.

Normally you will leave this blank and rclone will determine thecorrect root to use itself.

However you can set this to restrict rclone to a specific folderhierarchy or to access data within the 'Computers' tab on the driveweb interface (where files from Google's Backup and Sync desktopprogram go).

In order to do this you will have to find the Folder ID of thedirectory you wish rclone to display. This will be the last segmentof the URL when you open the relevant folder in the drive webinterface.

So if the folder you want rclone to use has a URL which looks likehttps://drive.google.com/drive/folders/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHChin the browser, then you use 1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh asthe root_folder_id in the config.

NB folders under the 'Computers' tab seem to be read only (drivegives a 500 error) when using rclone.

There doesn't appear to be an API to discover the folder IDs of the'Computers' tab - please contact us if you know otherwise!

Note also that rclone can't access any data under the 'Backups' tab onthe google drive web interface yet.

Service Account support

You can set up rclone with Google Drive in an unattended mode,i.e. not tied to a specific end-user Google account. This is usefulwhen you want to synchronise files onto machines that don't haveactively logged-in users, for example build machines.

To use a Service Account instead of OAuth2 token flow, enter the pathto your Service Account credentials at the service_account_fileprompt during rclone config and rclone won't use the browser basedauthentication flow. If you'd rather stuff the contents of thecredentials file into the rclone config file, you can setservice_account_credentials with the actual contents of the fileinstead, or set the equivalent environment variable.

Use case - Google Apps/G-suite account and individual Drive

Let's say that you are the administrator of a Google Apps (old) orG-suite account.The goal is to store data on an individual's Drive account, who ISa member of the domain.We'll call the domain example.com, and the user[email protected].

There's a few steps we need to go through to accomplish this:

1. Create a service account for example.com
  • To create a service account and obtain its credentials, go to theGoogle Developer Console.
  • You must have a project - create one if you don't.
  • Then go to 'IAM & admin' -> 'Service Accounts'.
  • Use the 'Create Credentials' button. Fill in 'Service account name'with something that identifies your client. 'Role' can be empty.
  • Tick 'Furnish a new private key' - select 'Key type JSON'.
  • Tick 'Enable G Suite Domain-wide Delegation'. This option makes'impersonation' possible, as documented here:Delegating domain-wide authority to the service account
  • These credentials are what rclone will use for authentication.If you ever need to remove access, press the 'Delete serviceaccount key' button.
2. Allowing API access to example.com Google Drive
  • Go to example.com's admin console
  • Go into 'Security' (or use the search bar)
  • Select 'Show more' and then 'Advanced settings'
  • Select 'Manage API client access' in the 'Authentication' section
  • In the 'Client Name' field enter the service account's'Client ID' - this can be found in the Developer Console under'IAM & Admin' -> 'Service Accounts', then 'View Client ID' forthe newly created service account.It is a ~21 character numerical string.
  • In the next field, 'One or More API Scopes', enterhttps://www.googleapis.com/auth/driveto grant access to Google Drive specifically.
3. Configure rclone, assuming a new install
4. Verify that it's working
  • rclone -v --drive-impersonate [email protected] lsf gdrive:backup
  • The arguments do:
    • -v - verbose logging
    • --drive-impersonate [email protected] - this is what doesthe magic, pretending to be user foo.
    • lsf - list files in a parsing friendly way
    • gdrive:backup - use the remote called gdrive, work inthe folder named backup.

Note: in case you configured a specific root folder on gdrive and rclone is unable to access the contents of that folder when using --drive-impersonate, do this instead:

  • in the gdrive web interface, share your root folder with the user/email of the new Service Account you created/selected at step #1
  • use rclone without specifying the --drive-impersonate option, like this:rclone -v [email protected] lsf gdrive:backup

Shared drives (team drives)

If you want to configure the remote to point to a Google Shared Drive(previously known as Team Drives) then answer y to the questionConfigure this as a Shared Drive (Team Drive)?.

This will fetch the list of Shared Drives from google and allow you toconfigure which one you want to use. You can also type in a SharedDrive ID if you prefer.

For example:

--fast-list

This remote supports --fast-list which allows you to use fewertransactions in exchange for more memory. See the rclonedocs for more details.

It does this by combining multiple list calls into a single API request.

This works by combining many '%s' in parents filters into one expression.To list the contents of directories a, b and c, the following requests will be send by the regular List function:

These can now be combined into a single request:

The implementation of ListR will put up to 50 parents filters into one request.It will use the --checkers value to specify the number of requests to run in parallel.

In tests, these batch requests were up to 20x faster than the regular method.Running the following command against different sized folders gives:

small folder (220 directories, 700 files):

  • without --fast-list: 38s
  • with --fast-list: 10s

large folder (10600 directories, 39000 files):

  • without --fast-list: 22:05 min
  • with --fast-list: 58s

Modified time

Google drive stores modification times accurate to 1 ms.

Restricted filename characters

Only Invalid UTF-8 bytes will be replaced,as they can't be used in JSON strings.

In contrast to other backends, / can also be used in names and .or .. are valid names.

Revisions

Google drive stores revisions of files. When you upload a change toan existing file to google drive using rclone it will create a newrevision of that file.

Revisions follow the standard google policy which at time of writingwas

  • They are deleted after 30 days or 100 revisions (whatever comes first).
  • They do not count towards a user storage quota.

Deleting files

By default rclone will send all files to the trash when deletingfiles. If deleting them permanently is required then use the--drive-use-trash=false flag, or set the equivalent environmentvariable.

Shortcuts

In March 2020 Google introduced a new feature in Google Drive calleddrive shortcuts(API). Thesewill (by September 2020) replace the ability for files or folders tobe in multiple folders at once.

Shortcuts are files that link to other files on Google Drive somewhatlike a symlink in unix, except they point to the underlying file data(e.g. the inode in unix terms) so they don't break if the source isrenamed or moved about.

Be default rclone treats these as follows.

For shortcuts pointing to files:

  • When listing a file shortcut appears as the destination file.
  • When downloading the contents of the destination file is downloaded.
  • When updating shortcut file with a non shortcut file, the shortcut is removed then a new file is uploaded in place of the shortcut.
  • When server-side moving (renaming) the shortcut is renamed, not the destination file.
  • When server-side copying the shortcut is copied, not the contents of the shortcut.
  • When deleting the shortcut is deleted not the linked file.
  • When setting the modification time, the modification time of the linked file will be set.

For shortcuts pointing to folders:

  • When listing the shortcut appears as a folder and that folder will contain the contents of the linked folder appear (including any sub folders)
  • When downloading the contents of the linked folder and sub contents are downloaded
  • When uploading to a shortcut folder the file will be placed in the linked folder
  • When server-side moving (renaming) the shortcut is renamed, not the destination folder
  • When server-side copying the contents of the linked folder is copied, not the shortcut.
  • When deleting with rclone rmdir or rclone purge the shortcut is deleted not the linked folder.
  • NB When deleting with rclone remove or rclone mount the contents of the linked folder will be deleted.

The rclone backend command can be used to create shortcuts.

Shortcuts can be completely ignored with the --drive-skip-shortcuts flagor the corresponding skip_shortcuts configuration setting.

Emptying trash

If you wish to empty your trash you can use the rclone cleanup remote:command which will permanently delete all your trashed files. This commanddoes not take any path arguments.

Note that Google Drive takes some time (minutes to days) to empty thetrash even though the command returns within a few seconds. No outputis echoed, so there will be no confirmation even using -v or -vv.

Quota information

To view your current quota you can use the rclone about remote:command which will display your usage limit (quota), the usage in GoogleDrive, the size of all files in the Trash and the space used by otherGoogle services such as Gmail. This command does not take any patharguments.

Import/Export of google documents

Google documents can be exported from and uploaded to Google Drive.

When rclone downloads a Google doc it chooses a format to downloaddepending upon the --drive-export-formats setting.By default the export formats are docx,xlsx,pptx,svg which are asensible default for an editable document.

When choosing a format, rclone runs down the list provided in orderand chooses the first file format the doc can be exported as from thelist. If the file can't be exported to a format on the formats list,then rclone will choose a format from the default list.

If you prefer an archive copy then you might use --drive-export-formats pdf, or if you prefer openoffice/libreoffice formats you might use--drive-export-formats ods,odt,odp.

Note that rclone adds the extension to the google doc, so if it iscalled My Spreadsheet on google docs, it will be exported as My Spreadsheet.xlsx or My Spreadsheet.pdf etc.

When importing files into Google Drive, rclone will convert allfiles with an extension in --drive-import-formats to theirassociated document type.rclone will not convert any files by default, since the conversionis lossy process.

The conversion must result in a file with the same extension whenthe --drive-export-formats rules are applied to the uploaded document.

Here are some examples for allowed and prohibited conversions.

export-formatsimport-formatsUpload ExtDocument ExtAllowed
odtodtodtodtYes
odtdocx,odtodtodtYes
docxdocxdocxYes
odtodtdocxNo
odt,docxdocx,odtdocxodtNo
docx,odtdocx,odtdocxdocxYes
docx,odtdocx,odtodtdocxNo

This limitation can be disabled by specifying --drive-allow-import-name-change.When using this flag, rclone can convert multiple files types resultingin the same document type at once, e.g. with --drive-import-formats docx,odt,txt,all files having these extension would result in a document represented as a docx file.This brings the additional risk of overwriting a document, if multiple fileshave the same stem. Many rclone operations will not handle this name changein any way. They assume an equal name when copying files and might copy thefile again or delete them when the name changes.

Here are the possible export extensions with their corresponding mime types.Most of these can also be used for importing, but there more that are notlisted here. Some of these additional ones might only be available whenthe operating system provides the correct MIME type entries.

This list can be changed by Google Drive at any time and might notrepresent the currently available conversions.

ExtensionMime TypeDescription
csvtext/csvStandard CSV format for Spreadsheets
docxapplication/vnd.openxmlformats-officedocument.wordprocessingml.documentMicrosoft Office Document
epubapplication/epub+zipE-book format
htmltext/htmlAn HTML Document
jpgimage/jpegA JPEG Image File
jsonapplication/vnd.google-apps.script+jsonJSON Text Format
odpapplication/vnd.oasis.opendocument.presentationOpenoffice Presentation
odsapplication/vnd.oasis.opendocument.spreadsheetOpenoffice Spreadsheet
odsapplication/x-vnd.oasis.opendocument.spreadsheetOpenoffice Spreadsheet
odtapplication/vnd.oasis.opendocument.textOpenoffice Document
pdfapplication/pdfAdobe PDF Format
pngimage/pngPNG Image Format
pptxapplication/vnd.openxmlformats-officedocument.presentationml.presentationMicrosoft Office Powerpoint
rtfapplication/rtfRich Text Format
svgimage/svg+xmlScalable Vector Graphics Format
tsvtext/tab-separated-valuesStandard TSV format for spreadsheets
txttext/plainPlain Text
xlsxapplication/vnd.openxmlformats-officedocument.spreadsheetml.sheetMicrosoft Office Spreadsheet
zipapplication/zipA ZIP file of HTML, Images CSS

Google documents can also be exported as link files. These files willopen a browser window for the Google Docs website of that documentwhen opened. The link file extension has to be specified as a--drive-export-formats parameter. They will match all availableGoogle Documents.

ExtensionDescriptionOS Support
desktopfreedesktop.org specified desktop entryLinux
link.htmlAn HTML Document with a redirectAll
urlINI style link filemacOS, Windows
weblocmacOS specific XML formatmacOS

Standard Options

Here are the standard options specific to drive (Google Drive).

--drive-client-id

Google Application Client IdSetting your own is recommended.See https://rclone.org/drive/#making-your-own-client-id for how to create your own.If you leave this blank, it will use an internal key which is low performance.

  • Config: client_id
  • Env Var: RCLONE_DRIVE_CLIENT_ID
  • Type: string
  • Default: '

--drive-client-secret

OAuth Client SecretLeave blank normally.

  • Config: client_secret
  • Env Var: RCLONE_DRIVE_CLIENT_SECRET
  • Type: string
  • Default: '

--drive-scope

Scope that rclone should use when requesting access from drive.

  • Config: scope
  • Env Var: RCLONE_DRIVE_SCOPE
  • Type: string
  • Default: '
  • Examples:
    • 'drive'
      • Full access all files, excluding Application Data Folder.
    • 'drive.readonly'
      • Read-only access to file metadata and file contents.
    • 'drive.file'
      • Access to files created by rclone only.
      • These are visible in the drive website.
      • File authorization is revoked when the user deauthorizes the app.
    • 'drive.appfolder'
      • Allows read and write access to the Application Data folder.
      • This is not visible in the drive website.
    • 'drive.metadata.readonly'
      • Allows read-only access to file metadata but
      • does not allow any access to read or download file content.

--drive-root-folder-id

ID of the root folderLeave blank normally.

Fill in to access 'Computers' folders (see docs), or for rclone to usea non root folder as its starting point.

  • Config: root_folder_id
  • Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
  • Type: string
  • Default: '

--drive-service-account-file

Service Account Credentials JSON file pathLeave blank normally.Needed only if you want use SA instead of interactive login.

Leading ~ will be expanded in the file name as will environment variables such as ${RCLONE_CONFIG_DIR}.

  • Config: service_account_file
  • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
  • Type: string
  • Default: '

--drive-alternate-export

Deprecated: no longer needed

  • Config: alternate_export
  • Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
  • Type: bool
  • Default: false

Advanced Options

Here are the advanced options specific to drive (Google Drive).

--drive-token

OAuth Access Token as a JSON blob.

  • Config: token
  • Env Var: RCLONE_DRIVE_TOKEN
  • Type: string
  • Default: '

--drive-auth-url

Auth server URL.Leave blank to use the provider defaults.

  • Config: auth_url
  • Env Var: RCLONE_DRIVE_AUTH_URL
  • Type: string
  • Default: '

--drive-token-url

Token server url.Leave blank to use the provider defaults.

  • Config: token_url
  • Env Var: RCLONE_DRIVE_TOKEN_URL
  • Type: string
  • Default: '

--drive-service-account-credentials

Service Account Credentials JSON blobLeave blank normally.Needed only if you want use SA instead of interactive login.

  • Config: service_account_credentials
  • Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
  • Type: string
  • Default: '

--drive-team-drive

ID of the Shared Drive (Team Drive)

  • Config: team_drive
  • Env Var: RCLONE_DRIVE_TEAM_DRIVE
  • Type: string
  • Default: '

--drive-auth-owner-only

Only consider files owned by the authenticated user.

  • Config: auth_owner_only
  • Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
  • Type: bool
  • Default: false

--drive-use-trash

Send files to the trash instead of deleting permanently.Defaults to true, namely sending files to the trash.Use --drive-use-trash=false to delete files permanently instead.

  • Config: use_trash
  • Env Var: RCLONE_DRIVE_USE_TRASH
  • Type: bool
  • Default: true

--drive-skip-gdocs

Skip google documents in all listings.If given, gdocs practically become invisible to rclone.

  • Config: skip_gdocs
  • Env Var: RCLONE_DRIVE_SKIP_GDOCS
  • Type: bool
  • Default: false

--drive-skip-checksum-gphotos

Skip MD5 checksum on Google photos and videos only.

Use this if you get checksum errors when transferring Google photos orvideos.

Setting this flag will cause Google photos and videos to return ablank MD5 checksum.

Google photos are identified by being in the 'photos' space.

Corrupted checksums are caused by Google modifying the image/video butnot updating the checksum.

  • Config: skip_checksum_gphotos
  • Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
  • Type: bool
  • Default: false

--drive-shared-with-me

Drive

Only show files that are shared with me.

Instructs rclone to operate on your 'Shared with me' folder (whereGoogle Drive lets you access the files and folders others have sharedwith you).

This works both with the 'list' (lsd, lsl, etc.) and the 'copy'commands (copy, sync, etc.), and with all other commands too.

  • Config: shared_with_me
  • Env Var: RCLONE_DRIVE_SHARED_WITH_ME
  • Type: bool
  • Default: false

--drive-trashed-only

Only show files that are in the trash.This will show trashed files in their original directory structure.

  • Config: trashed_only
  • Env Var: RCLONE_DRIVE_TRASHED_ONLY
  • Type: bool
  • Default: false

--drive-starred-only

Only show files that are starred.

  • Config: starred_only
  • Env Var: RCLONE_DRIVE_STARRED_ONLY
  • Type: bool
  • Default: false

--drive-formats

Deprecated: see export_formats

  • Config: formats
  • Env Var: RCLONE_DRIVE_FORMATS
  • Type: string
  • Default: '

--drive-export-formats

Comma separated list of preferred formats for downloading Google docs.

  • Config: export_formats
  • Env Var: RCLONE_DRIVE_EXPORT_FORMATS
  • Type: string
  • Default: 'docx,xlsx,pptx,svg'

--drive-import-formats

Comma separated list of preferred formats for uploading Google docs.

  • Config: import_formats
  • Env Var: RCLONE_DRIVE_IMPORT_FORMATS
  • Type: string
  • Default: '

--drive-allow-import-name-change

Allow the filetype to change when uploading Google docs (e.g. file.doc to file.docx). This will confuse sync and reupload every time.

  • Config: allow_import_name_change
  • Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
  • Type: bool
  • Default: false

--drive-use-created-date

Use file created date instead of modified date.,

Useful when downloading data and you want the creation date used inplace of the last modified date.

WARNING: This flag may have some unexpected consequences.

When uploading to your drive all files will be overwritten unless theyhaven't been modified since their creation. And the inverse will occurwhile downloading. This side effect can be avoided by using the'--checksum' flag.

This feature was implemented to retain photos capture date as recordedby google photos. You will first need to check the 'Create a GooglePhotos folder' option in your google drive settings. You can then copyor move the photos locally and use the date the image was taken(created) set as the modification date.

  • Config: use_created_date
  • Env Var: RCLONE_DRIVE_USE_CREATED_DATE
  • Type: bool
  • Default: false

--drive-use-shared-date

Use date file was shared instead of modified date.

Note that, as with '--drive-use-created-date', this flag may haveunexpected consequences when uploading/downloading files.

If both this flag and '--drive-use-created-date' are set, the createddate is used.

  • Config: use_shared_date
  • Env Var: RCLONE_DRIVE_USE_SHARED_DATE
  • Type: bool
  • Default: false

--drive-list-chunk

Size of listing chunk 100-1000. 0 to disable.

  • Config: list_chunk
  • Env Var: RCLONE_DRIVE_LIST_CHUNK
  • Type: int
  • Default: 1000

--drive-impersonate

Impersonate this user when using a service account.

  • Config: impersonate
  • Env Var: RCLONE_DRIVE_IMPERSONATE
  • Type: string
  • Default: '

--drive-upload-cutoff

Cutoff for switching to chunked upload

  • Config: upload_cutoff
  • Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
  • Type: SizeSuffix
  • Default: 8M

--drive-chunk-size

Upload chunk size. Must a power of 2 >= 256k.

Making this larger will improve performance, but note that each chunkis buffered in memory one per transfer.

Reducing this will reduce memory usage but decrease performance.

  • Config: chunk_size
  • Env Var: RCLONE_DRIVE_CHUNK_SIZE
  • Type: SizeSuffix
  • Default: 8M

--drive-acknowledge-abuse

Set to allow files which return cannotDownloadAbusiveFile to be downloaded.

If downloading a file returns the error 'This file has been identifiedas malware or spam and cannot be downloaded' with the error code'cannotDownloadAbusiveFile' then supply this flag to rclone toindicate you acknowledge the risks of downloading the file and rclonewill download it anyway.

  • Config: acknowledge_abuse
  • Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
  • Type: bool
  • Default: false

--drive-keep-revision-forever

Keep new head revision of each file forever.

  • Config: keep_revision_forever
  • Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
  • Type: bool
  • Default: false

--drive-size-as-quota

Show sizes as storage quota usage, not actual size.

Show the size of a file as the storage quota used. This is thecurrent version plus any older versions that have been set to keepforever.

WARNING: This flag may have some unexpected consequences.

It is not recommended to set this flag in your config - therecommended usage is using the flag form --drive-size-as-quota whendoing rclone ls/lsl/lsf/lsjson/etc only.

If you do use this flag for syncing (not recommended) then you willneed to use --ignore size also.

  • Config: size_as_quota
  • Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
  • Type: bool
  • Default: false

--drive-v2-download-min-size

If Object's are greater, use drive v2 API to download.

  • Config: v2_download_min_size
  • Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
  • Type: SizeSuffix
  • Default: off

--drive-pacer-min-sleep

Minimum time to sleep between API calls.

  • Config: pacer_min_sleep
  • Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
  • Type: Duration
  • Default: 100ms

--drive-pacer-burst

Number of API calls to allow without sleeping.

  • Config: pacer_burst
  • Env Var: RCLONE_DRIVE_PACER_BURST
  • Type: int
  • Default: 100

--drive-server-side-across-configs

Allow server-side operations (e.g. copy) to work across different drive configs.

This can be useful if you wish to do a server-side copy between twodifferent Google drives. Note that this isn't enabled by defaultbecause it isn't easy to tell if it will work between any twoconfigurations.

  • Config: server_side_across_configs
  • Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
  • Type: bool
  • Default: false

--drive-disable-http2

Disable drive using http2

There is currently an unsolved issue with the google drive backend andHTTP/2. HTTP/2 is therefore disabled by default for the drive backendbut can be re-enabled here. When the issue is solved this flag willbe removed.

See: https://github.com/rclone/rclone/issues/3631

  • Config: disable_http2
  • Env Var: RCLONE_DRIVE_DISABLE_HTTP2
  • Type: bool
  • Default: true

--drive-stop-on-upload-limit

Make upload limit errors be fatal

At the time of writing it is only possible to upload 750GB of data toGoogle Drive a day (this is an undocumented limit). When this limit isreached Google Drive produces a slightly different error message. Whenthis flag is set it causes these errors to be fatal. These will stopthe in-progress sync.

Google Home Raspberry Pi

Note that this detection is relying on error message strings whichGoogle don't document so it may break in the future.

See: https://github.com/rclone/rclone/issues/3857

  • Config: stop_on_upload_limit
  • Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
  • Type: bool
  • Default: false

--drive-stop-on-download-limit

Make download limit errors be fatal

At the time of writing it is only possible to download 10TB of data fromGoogle Drive a day (this is an undocumented limit). When this limit isreached Google Drive produces a slightly different error message. Whenthis flag is set it causes these errors to be fatal. These will stopthe in-progress sync.

Note that this detection is relying on error message strings whichGoogle don't document so it may break in the future.

  • Config: stop_on_download_limit
  • Env Var: RCLONE_DRIVE_STOP_ON_DOWNLOAD_LIMIT
  • Type: bool
  • Default: false

--drive-skip-shortcuts

If set skip shortcut files

Normally rclone dereferences shortcut files making them appear as ifthey are the original file (see the shortcuts section).If this flag is set then rclone will ignore shortcut files completely.

  • Config: skip_shortcuts
  • Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS
  • Type: bool
  • Default: false

--drive-encoding

This sets the encoding for the backend.

See: the encoding section in the overview for more info.

  • Config: encoding
  • Env Var: RCLONE_DRIVE_ENCODING
  • Type: MultiEncoder
  • Default: InvalidUtf8

Backend commands

Here are the commands specific to the drive backend.

Run them with

The help below will explain what arguments each command takes.

See the 'rclone backend' command for moreinfo on how to pass options and arguments.

These can be run on a running backend using the rc commandbackend/command.

get

Get command for fetching the drive config parameters

This is a get command which will be used to fetch the various drive config parameters

Usage Examples:

Options:

  • 'chunk_size': show the current upload chunk size
  • 'service_account_file': show the current service account file

set

Set command for updating the drive config parameters

This is a set command which will be used to update the various drive config parameters

Usage Examples:

Options:

  • 'chunk_size': update the current upload chunk size
  • 'service_account_file': update the current service account file

shortcut

Create shortcuts from files or directories

This command creates shortcuts from files or directories.

Usage:

In the first example this creates a shortcut from the 'source_item'which can be a file or a directory to the 'destination_shortcut'. The'source_item' and the 'destination_shortcut' should be relative pathsfrom 'drive:'

In the second example this creates a shortcut from the 'source_item'relative to 'drive:' to the 'destination_shortcut' relative to'drive2:'. This may fail with a permission error if the userauthenticated with 'drive2:' can't read files from 'drive:'.

Options:

  • 'target': optional target remote for the shortcut destination

drives

List the Shared Drives available to this account

This command lists the Shared Drives (Team Drives) available to thisaccount.

Usage:

This will return a JSON list of objects like this

untrash

Untrash files and directories

This command untrashes all the files and directories in the directorypassed in recursively.

Usage:

This takes an optional directory to trash which make this easier touse via the API.

Use the -i flag to see what would be restored before restoring it.

Result:

copyid

Copy files by ID

This command copies files by ID

Usage:

It copies the drive file with ID given to the path (an rclone path whichwill be passed internally to rclone copyto). The ID and path pairs can berepeated.

The path should end with a / to indicate copy the file as named tothis directory. If it doesn't end with a / then the last pathcomponent will be used as the file name.

Raspberry Pi Stream Video

If the destination is a drive backend then server-side copying will beattempted if possible.

Use the -i flag to see what would be copied before copying.

Limitations

Drive has quite a lot of rate limiting. This causes rclone to belimited to transferring about 2 files per second only. Individualfiles may be transferred much faster at 100s of MBytes/s but lots ofsmall files can take a long time.

Server side copies are also subject to a separate rate limit. If yousee User rate limit exceeded errors, wait at least 24 hours and retry.You can disable server-side copies with --disable copy to downloadand upload the files if you prefer.

Limitations of Google Docs

Google docs will appear as size -1 in rclone ls and as size 0 inanything which uses the VFS layer, e.g. rclone mount, rclone serve.

This is because rclone can't find out the size of the Google docswithout downloading them.

Google docs will transfer correctly with rclone sync, rclone copyetc as rclone knows to ignore the size when doing the transfer.

However an unfortunate consequence of this is that you may not be ableto download Google docs using rclone mount. If it doesn't work youwill get a 0 sized file. If you try again the doc may gain itscorrect size and be downloadable. Whether it will work on not dependson the application accessing the mount and the OS you are running -experiment to find out if it does work for you!

Duplicated files

Raspberry Pi File Server

Sometimes, for no reason I've been able to track down, drive willduplicate a file that rclone uploads. Drive unlike all the otherremotes can have duplicated files.

Duplicated files cause problems with the syncing and you will seemessages in the log about duplicates.

Use rclone dedupe to fix duplicated files.

Note that this isn't just a problem with rclone, even Google Photos onAndroid duplicates files on drive sometimes.

Rclone appears to be re-copying files it shouldn't

Raspberry Pi File System

The most likely cause of this is the duplicated file issue above - runrclone dedupe and check your logs for duplicate object or directorymessages.

This can also be caused by a delay/caching on google drive's end whencomparing directory listings. Specifically with team drives used incombination with --fast-list. Files that were uploaded recently maynot appear on the directory list sent to rclone when using --fast-list.

Waiting a moderate period of time between attempts (estimated to beapproximately 1 hour) and/or not using --fast-list both seem to beeffective in preventing the problem.

Making your own client_id

When you use rclone with Google drive in its default configuration youare using rclone's client_id. This is shared between all the rcloneusers. There is a global rate limit on the number of queries persecond that each client_id can do set by Google. rclone already has ahigh quota and I will continue to make sure it is high enough bycontacting Google.

It is strongly recommended to use your own client ID as the default rclone ID is heavily used. If you have multiple services running, it is recommended to use an API key for each service. The default Google quota is 10 transactions per second so it is recommended to stay under that number as if you use more than that, it will cause rclone to rate limit and make things slower.

Here is how to create your own Google Drive client ID for rclone:

  1. Log into the Google APIConsole with your Googleaccount. It doesn't matter what Google account you use. (It need notbe the same account as the Google Drive you want to access)

  2. Select a project or create a new project.

  3. Under 'ENABLE APIS AND SERVICES' search for 'Drive', and enable the'Google Drive API'.

  4. Click 'Credentials' in the left-side panel (not 'Createcredentials', which opens the wizard), then 'Create credentials'

  5. If you already configured an 'Oauth Consent Screen', then skipto the next step; if not, click on 'CONFIGURE CONSENT SCREEN' button(near the top right corner of the right panel), then select 'External'and click on 'CREATE'; on the next screen, enter an 'Application name'('rclone' is OK) then click on 'Save' (all other data is optional).Click again on 'Credentials' on the left panel to go back to the'Credentials' screen.

(PS: if you are a GSuite user, you could also select 'Internal' insteadof 'External' above, but this has not been tested/documented so far).

  1. Click on the '+ CREATE CREDENTIALS' button at the top of the screen,then select 'OAuth client ID'.

  2. Choose an application type of 'Desktop app' if you using a Google account or 'Other' ifyou using a GSuite account and click 'Create'. (the default name is fine)

  3. It will show you a client ID and client secret. Use these valuesin rclone config to add a new remote or edit an existing remote.

Be aware that, due to the 'enhanced security' recently introduced byGoogle, you are theoretically expected to 'submit your app for verification'and then wait a few weeks(!) for their response; in practice, you can go rightahead and use the client ID and client secret with rclone, the only issue willbe a very scary confirmation screen shown when you connect via your browserfor rclone to be able to get its token-id (but as this only happens duringthe remote configuration, it's not such a big deal).

(Thanks to @balazer on github for these instructions.)

Sometimes, creation of an OAuth consent in Google API Console fails due to an error message“The request failed because changes to one of the field of the resource is not supported”.As a convenient workaround, the necessary Google Drive API key can be created on thePython Quickstart page.Just push the Enable the Drive API button to receive the Client ID and Secret.Note that it will automatically create a new project in the API Console.