How to Read Photos for Deep Learning on Google Cloud Platform Compute Engine VM Instance, Through Jupyter Lab


I've set up a project on GCP with a Compute Engine VM and Storage Bucket.
Access Scopes set to Allow full access to all Cloud APIs Have set a default Region and Zone I believe I have completed SSH authorization but not 100% sure given the documentation I've read.

I have ~100,000 photos on both my local machine that I managed to upload to a bucket and am now trying to run my algo using the VM connected jupyter lab. My first thought was to create a list object from the bucket object. After 90 min, I tried to do the same but with my local folder. After failing at that, I attempted to mount the bucket on the VM instance though, I failed at that too (i'd rather not go this rout anyway since I'd have to copy the files which seems nuts).

i've used this with different commands !gsutil ls gs://my_bucket

from import storage

bucket_name = "my_bucket"

storage_client = storage.Client()

bucket = storage_client.get_bucket(bucket_name)

I'm new to all this, have read through more GCP documentation pages than I can count and you would be disgusted with me about how much time I've given this...please help!


Posted 2019-09-17T00:58:43.333

Reputation: 51

No answers