Immutable to mutable bucket

TagoIO Analysis and Actions
1

We recently had to add an additional variable to be used for our solution. The initial setup is using an immutable bucket with a retention of 1 year. However, with the additional variable we need to change strategy. The 2 variables need to have different retention periods.

  1. Is our only option to change back from immutable to mutable and so manage the retentions from an analysis.

  2. If so, how to transfer from immutable to mutable without emptying the buckets.

  3. Or can we exclude variables from retention.

Thanks,

Vincent

2

Hi,

Given your requirement to have two variables with different retention periods within the same bucket, it’s important to note that the immutable bucket’s retention policy applies to the entire bucket, not to individual variables. This means that all data within the bucket will be subject to the same retention period.

To achieve different retention periods for two variables, you would indeed need to consider using mutable buckets, as these do not have a built-in data retention feature and allow for editing or deleting data as needed. This flexibility could enable you to manage data retention through custom Analysis scripts, where you could implement logic to delete or archive data based on your specific requirements for each variable.

To transfer data from an immutable to a mutable bucket, you could use Analysis scripts to read data from the immutable bucket and write it to a new mutable bucket. This process would involve programmatically accessing the data in the immutable bucket, possibly filtering it as per your requirements, and then inserting it into the mutable bucket. However, this approach requires careful planning to ensure data integrity and to manage the transfer process efficiently.

3

Exporting data from an immutable bucket and migrating it to a new mutable bucket involves several steps. Here’s a guideline to help you through the process:

Step 1: Export Data from Immutable Bucket

  1. Access the Bucket Module: Navigate to the Buckets section in your TagoIO dashboard.
  2. Select Your Immutable Bucket: Look for the bucket marked as Immutable. This is the bucket from which you want to export data.
  3. Use Chunk Management for Export: Since you’re dealing with an immutable bucket, you’ll need to utilize the Chunk Management feature. This feature is specifically designed for Optimized Device Data (Immutable) buckets. Chunk Management allows you to manage data chunks based on the device creation timeframe.
  4. Export Data: In the Chunk Management section, you can transfer chunks to TagoIO Files in CSV format for backup. Customize the file address using variables such as $DEVICE$, $CHUNK$, $FROM$, $TO$. Ensure you include headers in your CSV files if needed. Remember, exporting data consumes Data Output transactions.

Step 2: Create a New Mutable Bucket

  1. Navigate to Buckets: Go to the Buckets list in your TagoIO dashboard.
  2. Create a New Bucket: Click on the option to create a new bucket. During the creation process, ensure you select the mutable bucket option. Mutable buckets allow editing or deleting data as needed and are designed for storing and manipulating configurable variables.

Step 3: Import Data to the New Mutable Bucket

  1. Prepare Your Data: Ensure the data exported from the immutable bucket is in a format compatible with TagoIO’s data import mechanisms. CSV is a commonly used format for this purpose.
  2. Import Data: Currently, direct import options into buckets through the UI are limited. However, you can use TagoIO’s API or Analysis scripts to programmatically upload the data from your CSV files into the new mutable bucket. This might involve writing a script that reads your exported data and uses TagoIO’s API to insert the data into the new bucket.

Step 4: Verify Data Integrity

  1. Check Imported Data: After importing the data into the new mutable bucket, verify that all data points have been correctly transferred and that the data integrity is maintained.
  2. Adjust as Necessary: If you notice any discrepancies or issues with the imported data, you may need to adjust your import script or process accordingly.

For more detailed information on managing buckets and data within TagoIO, you can refer to the following resources:

This guideline should help you successfully export data from an immutable bucket and migrate it to a new mutable bucket within TagoIO.