Uploading images folder from my system into Google Colab

20

14

I want to train a deep learning model on a dataset containing around 3000 images. Since the dataset is huge, I want to use Google colab since it's GPU supported. How do I upload this full image folder into my notebook and use it?

chatbot_chakra

Posted 2018-03-23T18:52:28.867

Reputation: 311

Answers

14

Method 1 :

  1. zip the file
  2. Upload the zipped file, there is an Upload button under the Files Section.
  3. Unzip it using the command on colab : !unzip level_1_test.zip

Method 2 :

  1. upload the zip file to the google drive account.
  2. The only difference is in step 2 where in place of the GUI upload option you can run the google code_snippets to upload download your zip file from the google drive to Colab account .
  3. Unzip it using the command on colab : !unzip level_1_test.zip

Srijeet

Posted 2018-03-23T18:52:28.867

Reputation: 141

Thanks a lot! Subjectively - method 2 is faster. Also the easier way to download a file from drive to colab is described here in the top answer https://stackoverflow.com/questions/48735600/file-download-from-google-drive-to-colaboratory

– Kirill Starostin – 2020-06-08T18:31:22.067

8

The best bet would be to upload the images as a zip file to your Google drive and then access it through Google Colab (GC)

  1. Zip the image folder
  2. Upload the zip file to your Google drive
  3. Turn to GC to authorise and mount your Google drive

    from google.colab import drive
    drive.mount('/content/drive')
    
  4. Follow the link and paste the code to your GC notebook

  5. Unzip the file from GC

    !unzip -uq "/content/drive/My Drive/PATH_TO_ZIP" -d "/content/drive/My Drive/PATH_TO_OUTPUT"
    
  6. The files are now ready to use

Surya Palaniswamy

Posted 2018-03-23T18:52:28.867

Reputation: 81

2

You don't need to upload them if you have a download link ...( it would be faster if you can upload them all as either ways you have to do so.. So its better to upload them first and then download them in your notebook every-time you run it)

If you have a download link then just this

! wget <Link>

Else upload then to your drive and then just use the following

from google.colab import files

uploaded = files.upload()

##files.upload returns a dictionary of the files which were uploaded. The 
##dictionary is keyed by the file name, the value is the data which was 
##uploaded.

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn]))

Aditya

Posted 2018-03-23T18:52:28.867

Reputation: 2 128

0

I recommend you uploading a zip file containing your images to your drive and downloading the content from drive to Colab. Then you will be able to extract them. The code for uploading is here.

Media

Posted 2018-03-23T18:52:28.867

Reputation: 12 077

0

You can upload stuff to Google Drive and then download it from there on Colab. I've written some utils for that - see this notebook.

As to how upload files to Google Drive, Media's suggestion is useful - upload zipped image folder.

Jakub Bartczuk

Posted 2018-03-23T18:52:28.867

Reputation: 290

0

On Colab simply use:

!gdown --id file_id

For instance, if your Google drive share link for the file is:

https://drive.google.com/file/d/zz2Xs5Vriz6aF3V-Z22112yAj91c222fI1F/view?usp=sharing

use:

!gdown --id zz2Xs5Vriz6aF3V-Z22112yAj91c222fI1F

Rexcirus

Posted 2018-03-23T18:52:28.867

Reputation: 131