to create object in S3, I would like to suggest to transform the data to `parquet` which is faster than `csv`
Encapsulation = wrapping the data
The OOPs the usage of encapsulation is very normal and easy. It is important to access the value of the variable through the function.
In the following simple example, we can get the value of name using get_name() function.
Some real time example like the following can explain the real importance of the encapsulation.
Objective
To read the JSON file and save it to the variable
import jsonclass JSONReader:
def __init__(self, file_path: str):
self.file_path = file_path def read(self):
with open(self.book_path, 'r') as myfile:
data = myfile.read()
return json.loads(data)if __name__ == "__main__":
reader = JSONReader("/tmp/a.json")
print(reader.read())
The file context manager can read the file as string, then json.loads(<string>) will convert the string value to the JSON object
nice article and list of tools... i recommended this article to our QA team...
I am using Dagster for data pipeline. I am not sure whether its fit on this box...
