BigQuery cross-region replication: Improved data geo-redundancy

Ensuring Data Safety: BigQuery Cross-Region replication One of the most important factors in creating a robust cloud data lake architecture is geographic redundancy

[{"selector":"#anim-67ff9836-0b1d-4ed6-bc60-51d047c993c3 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(34.249999904429615%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-1a327873-ea61-4800-bbc2-215304880147","keyframes":[{"offset":0,"transform":"translate3d(0, -147.11979%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.29,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.45,"transform":"translate3d(0, -41.370084948%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.61,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.71,"transform":"translate3d(0, -14.064651924%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.8,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.85,"transform":"translate3d(0, -5.281600461%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.92,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.96,"transform":"translate3d(0, -2.2950687239999996%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":1,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"}],"delay":0,"duration":600,"fill":"both"}]

Within a dataset region, BigQuery already keeps copies of your data in two distinct Google Cloud zones. Symmetric dual writes are used for zone-to-zone replication in all regions

[{"selector":"#anim-014a6a6b-7368-493e-9d52-516ac0c10c4d [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(21.874999829338595%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-327e694c-a047-4bfa-b504-ed47dfd1b730","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4ffb5803-321d-467c-8129-f96c0a994e0d","keyframes":{"transform":["translate3d(0px, -117.28849%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

With the preview of cross-region dataset replication, Google are thrilled to advance this further by making it simple to duplicate any dataset including live updates across cloud regions

[{"selector":"#anim-a673ecd9-1a4e-4c24-8c5e-371f107d833b [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(28.93339755556636%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-d892cbb6-71d7-4286-b3a3-cecae399b8bc","keyframes":{"transform":["scale(1)","scale(1.5)","scale(0.95)","scale(1)"],"offset":[0,0.33,0.66,1]},"delay":0,"duration":1450,"easing":"ease-in-out","fill":"both","iterations":1}]

Two redundant zones are used to store the data within each region The Google Cloud network never experiences network outage

[{"selector":"#anim-f5b9eb12-6acd-49f0-879d-7de049d3c79a [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(35.937499914669296%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-0c7a4a39-ecdd-41f5-ba6c-d3adb578c30b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-cf2d18e4-f467-43b7-9ec4-511ea91239bf","keyframes":{"transform":["translate3d(0px, 137.28455%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

To accomplish this, use the BigQuery API or query settings to set the job location to the secondary region BigQuery automatically directs your queries to the primary replica’s location if you don’t provide one

[{"selector":"#anim-2490f818-d6a1-41b3-9ea2-327b269f93e7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(31.35190206075711%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-81aceb3b-f2d1-48cd-bd9e-b39b50b7c191","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b4a2ac3f-574f-404f-a482-3002231a9a53","keyframes":{"transform":["translate3d(0px, 112.47398%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

Google want to incorporate a console-based user interface for replica configuration and management In the unlikely event of a complete regional outage, Google will also provide a cross-region disaster recovery

[{"selector":"#anim-c449b604-1517-4ce4-9d39-50fa4acf1c71 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(35.937499914669296%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-5392d83a-ebef-42a4-a71e-56243036e308","keyframes":{"transform":["translate3d(-124.01434%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-25c88296-6a04-4c99-bfdf-9e549589b354","keyframes":{"transform":["rotateZ(-180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Google are thrilled to announce that BigQuery now supports the cross-region replication enabling you to improve geo-redundancy and accommodate use cases involving region movement

[{"selector":"#anim-888fb6e7-279e-49ae-bf1c-fc2538895e23 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-468217c2-94d2-4bdd-9601-cef56b2b76aa","keyframes":{"transform":["translate3d(-120.61856%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-14fcf459-9650-4b18-b81e-63a89ab63663","keyframes":{"transform":["rotateZ(-180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

The BigQuery cross-region dataset replication QuickStart has additional information regarding BigQuery and cross-region replication For more details visit govindhtech.com

[{"selector":"#anim-186596b4-c897-464d-880b-d30581ad0fc7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(31.35190206075711%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}]