{"id":7861,"date":"2025-12-12T14:14:58","date_gmt":"2025-12-12T13:14:58","guid":{"rendered":"https:\/\/businessintelligence.pl\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/"},"modified":"2026-09-16T15:28:35","modified_gmt":"2026-09-16T13:28:35","slug":"data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik","status":"publish","type":"post","link":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/","title":{"rendered":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik"},"content":{"rendered":"\nQlik has enhanced its Open Lakehouse platform with streaming data ingestion capabilities, real-time transformations, and extended integration with the Apache Iceberg ecosystem.\n\nThis is a response to the growing demands of teams building analytical and AI architectures on open formats, without vendor lock-in, and with full control over data in their own cloud.\n<h3><strong>What&#8217;s New in Qlik Open Lakehouse<\/strong><\/h3>\nThe most significant changes concern three areas: ingestion, transformations, and integrations.\n\nFirst: streaming ingestion. Qlik Open Lakehouse now supports streaming data ingestion from Apache Kafka, Amazon Kinesis, and Amazon S3 directly into managed Apache Iceberg tables. Data lands in the customer&#8217;s cloud (Amazon S3) without engaging data warehouse compute resources. Ingestion occurs on dedicated, cost-effective compute, which lowers costs and increases scalability.\n\nSecond: streaming transformations. Data can be processed on the fly \u2013 cleaned, filtered, normalized, and nested structures flattened \u2013 even before it reaches the target Iceberg table. This eliminates the need for additional ETL layers and shortens data availability time for downstream consumers.\n\nThird: automatic Iceberg optimization. Qlik manages file compaction and metadata updates, which is crucial for maintaining query performance with large volumes of streaming data. Users do not need to manually run OPTIMIZE or VACUUM operations \u2013 the platform does it automatically.\n<h3><strong>Integrations with the Iceberg Ecosystem<\/strong><\/h3>\nQlik has extended support for tools and platforms that work with Apache Iceberg:\n<ul>\n \t<li>Snowflake Open Catalog \u2013 in addition to AWS Glue, Qlik now also supports the Snowflake catalog<\/li>\n \t<li>Zero-copy mirroring to Databricks and Amazon Redshift \u2013 data in Iceberg is available in these platforms without duplication, analogous to existing mirroring to Snowflake<\/li>\n \t<li>Extended compatibility with Apache Spark \u2013 seamless access to current Iceberg tables<\/li>\n \t<li>Support for Amazon Athena, Amazon SageMaker Studio, Trino, Presto \u2013 data written once is queryable by multiple engines<\/li>\n<\/ul>\nZero-copy mirroring is a key feature for hybrid lakehouse-warehouse architectures. Data physically remains in Iceberg on S3 but is visible and accessible in Snowflake, Databricks, or Redshift without additional copies. This reduces storage costs and simplifies management.\n<h3><strong>Governance and Data Quality<\/strong><\/h3>\nQlik automatically applies governance to data in Open Lakehouse. Each Iceberg table has:\n<ul>\n \t<li>Data lineage \u2013 full traceability of data origin, from source to consumer<\/li>\n \t<li>Qlik Trust Score \u2013 an automatically calculated indicator of data quality and reliability<\/li>\n \t<li>Catalog services \u2013 automatic metadata catalog updates<\/li>\n \t<li>Data quality checks \u2013 data validation in pipelines<\/li>\n<\/ul>\nGovernance here is not an overlay or a separate tool \u2013 it is built into the Qlik Talend Cloud platform and operates at the level of pipelines, tables, and mirrored datasets.\n<h3><strong>Architecture and Deployment<\/strong><\/h3>\nQlik Open Lakehouse manages Apache Iceberg tables on Amazon S3 in the customer&#8217;s environment. Streaming pipelines write events and apply transformations as data flows. Compression and metadata updates occur automatically, maintaining performance at scale.\n\nData is written once, in the open Iceberg format, and is available to multiple analytical engines without duplication. This is a classic lakehouse architecture, but with management that eliminates typical problems: lack of optimization, catalog chaos, missing lineage, and lack of quality control.\n<h3><strong>Availability<\/strong><\/h3>\nThe new streaming ingestion and streaming transformations features are expected to be generally available in Q1 2026 for Qlik Talend Cloud customers. Support for Snowflake Open Catalog, extended compatibility with Apache Spark, and zero-copy mirroring to Databricks and Amazon Redshift will be rolled out in phases, starting from Q1 2026, with a regional schedule announced as features become available.\n\nFor teams building modern data architectures on Apache Iceberg, Qlik Open Lakehouse offers a managed platform with full governance, without sacrificing openness and control over data. This is a combination that is difficult to achieve in practice \u2013 and precisely why it is worth noting.\n","protected":false},"excerpt":{"rendered":"<p>Qlik has enhanced its Open Lakehouse platform with streaming data ingestion capabilities, real-time transformations, and extended integration with the Apache Iceberg ecosystem.<\/p>\n","protected":false},"author":1,"featured_media":7860,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[161],"tags":[],"class_list":["post-7861","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-talend"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik - Hogart Business Intelligence<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik - Hogart Business Intelligence\" \/>\n<meta property=\"og:description\" content=\"Qlik has enhanced its Open Lakehouse platform with streaming data ingestion capabilities, real-time transformations, and extended integration with the Apache Iceberg ecosystem.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/\" \/>\n<meta property=\"og:site_name\" content=\"Hogart Business Intelligence\" \/>\n<meta property=\"article:published_time\" content=\"2025-12-12T13:14:58+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-09-16T13:28:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1706\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"hogart.admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hogart.admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik - Hogart Business Intelligence","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/","og_locale":"en_US","og_type":"article","og_title":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik - Hogart Business Intelligence","og_description":"Qlik has enhanced its Open Lakehouse platform with streaming data ingestion capabilities, real-time transformations, and extended integration with the Apache Iceberg ecosystem.","og_url":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/","og_site_name":"Hogart Business Intelligence","article_published_time":"2025-12-12T13:14:58+00:00","article_modified_time":"2026-09-16T13:28:35+00:00","og_image":[{"width":2560,"height":1706,"url":"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg","type":"image\/jpeg"}],"author":"hogart.admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hogart.admin","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#article","isPartOf":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/"},"author":{"name":"hogart.admin","@id":"https:\/\/businessintelligence.pl\/en\/#\/schema\/person\/cfbfeb44076f67efe7db28adfb6e2370"},"headline":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik","datePublished":"2025-12-12T13:14:58+00:00","dateModified":"2026-09-16T13:28:35+00:00","mainEntityOfPage":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/"},"wordCount":602,"image":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#primaryimage"},"thumbnailUrl":"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg","articleSection":["Talend"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/","url":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/","name":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik - Hogart Business Intelligence","isPartOf":{"@id":"https:\/\/businessintelligence.pl\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#primaryimage"},"image":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#primaryimage"},"thumbnailUrl":"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg","datePublished":"2025-12-12T13:14:58+00:00","dateModified":"2026-09-16T13:28:35+00:00","author":{"@id":"https:\/\/businessintelligence.pl\/en\/#\/schema\/person\/cfbfeb44076f67efe7db28adfb6e2370"},"breadcrumb":{"@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#primaryimage","url":"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg","contentUrl":"https:\/\/businessintelligence.pl\/wp-content\/uploads\/2026\/01\/tlo-przechowywania-w-chmurze-projekt-sieci-biznesowej-scaled-1.jpg","width":2560,"height":1706},{"@type":"BreadcrumbList","@id":"https:\/\/businessintelligence.pl\/en\/data-lakehouse-architecture-streaming-ingestion-and-zero-copy-data-mirroring-with-qlik\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Strona g\u0142\u00f3wna","item":"https:\/\/businessintelligence.pl\/en\/"},{"@type":"ListItem","position":2,"name":"Data lakehouse architecture: streaming ingestion and zero-copy data mirroring with Qlik"}]},{"@type":"WebSite","@id":"https:\/\/businessintelligence.pl\/en\/#website","url":"https:\/\/businessintelligence.pl\/en\/","name":"Hogart Business Intelligence","description":"Polski partner Qlik posiadaj\u0105cy status Elite Solution Provider","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/businessintelligence.pl\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/businessintelligence.pl\/en\/#\/schema\/person\/cfbfeb44076f67efe7db28adfb6e2370","name":"hogart.admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/7e2a05e9a820d06d4bfc1cb3df775680738fb50b56d2f95632a5d5210f45f6b5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/7e2a05e9a820d06d4bfc1cb3df775680738fb50b56d2f95632a5d5210f45f6b5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7e2a05e9a820d06d4bfc1cb3df775680738fb50b56d2f95632a5d5210f45f6b5?s=96&d=mm&r=g","caption":"hogart.admin"},"sameAs":["https:\/\/businessintelligence.pl"]}]}},"_links":{"self":[{"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/posts\/7861","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/comments?post=7861"}],"version-history":[{"count":1,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/posts\/7861\/revisions"}],"predecessor-version":[{"id":7862,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/posts\/7861\/revisions\/7862"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/media\/7860"}],"wp:attachment":[{"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/media?parent=7861"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/categories?post=7861"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/businessintelligence.pl\/en\/wp-json\/wp\/v2\/tags?post=7861"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}