@B-M Here is the overview that I have in the plugin's backend settings screen:
To stay within free usage, we are going to save data to Cloud Storage and load those files into BigQuery instead of streaming data directly into BigQuery. Because of this, there are 2 different Cloud Functions that will be utilized.
As an overview, these are the steps that will be taken:
Cloud Function (HTTP): Receive JSON data from plugin. Save as JSON line-delimited file in Cloud Storage.
Cloud Storage: Bucket will temporarily hold generated data file.
Cloud Function (Storage trigger): Function is triggered when a new file is uploaded to storage bucket. Meta data will be read to know what table to load the data into, and the file will be loaded into BigQuery.
BigQuery: Holds data ready for reporting. Easy to connect to various visualization tools
Let me know if that's helpful. I could share the javascript code that I use for the functions, but I would have to remove some proprietary logic that I added to mine to support some other specific business needs.