StarRocks+极速统一的湖仓分析.pptx

S匧鸟ta絡r?R涸o廩c?k?s區

StarRcoksStarRocksStarRocks

01StarRcoks

?//?→????

SingleSourceofTruth:

StarRocksLakeHouseDatawarehouseDataLakeLakeHouseDataqualityPerformanceRealtime???GovernanceOpen?Singlesourceoftruth的ScalabilityCostefficiencyStarRocks?catalog??OpenTableFormat

02StarRcoks

LakeHouseStarRocksFESpark/FlinkComputeengineCatalogTableformatFileformatStorageHMSIcebergParquetHDFSObjectstorageORCHudiStarRocksFileGlue/DLFStarRocksTableStarRocksCN统一开放的Lakehouse架构，分层解耦设计Lakehouse?Catalog?HDFS

CBOOptimizerVectorengineComputeEngineMVExternalTableStatisticsTrinoCompatibleCatalogExternalCatalogUnifiedCatalogTableFormatHive/HudiMetadataCacheIcebergMetadataCacheIcebergColocate/BucketJoinFileFormatNativeReaderIOCoalesceLateMaterializationCost-BasedLateMaterializationDataCacheFooter/PageCache

DataCacheCacheBlocklevelDataCacheCache??+Hash??Cache务LakeBEExecutDataCacheBEExecutDataCacheBEExecutDataCacheBalancedDistributionGranularCachingScanRange0CN0CN1ConsistentHashScanRange1CN3CN2ScanRangeN

DataCache?DataCache12%Cache吐?ExcutionEngineDataCacheHDFS/S3ScanBusyScan

ParquetReaderColumnCol1Col2Col3IORowGroup务IOFooter/PageCache/Page/FooterIO

ParquetReader-Col2务Col1Col2Col1SKIPScanandFilterAND

ParquetReader-Col2Col1?MaskMaskCol1?去Enumeratetheorder

Hive/Hudi?cacheHMS/Glue?Table/Partition/ListFile?refreshexternaltabletbl_name??Cacherefresh_interval,cache_ttl,cacheListFileCacheOptimizerCoordinatorRefreshThreadRefreshTableRefreshManualRefreshAutoMetadataCacheTableC\aPcahretitionRefereshAsync\ExpireRefereshAsync\ExpireHMS\GlueStorage

Iceberg?IcebergV2TablePositionDeleteFilesColocate/BucketJoinbucketpartitiontransformJoinshuffleMemory/Diskmanifest,?MemoryMemoryDiskdatalayermetadatalayercurrentmetadatapointerIcebergCatalogcurrentdmbe1t,atdabatlea1pointermetadatafiles0metadatafiles0s1malnisitfestmafniliefestmafniliefestmalnisitfestmafniliefestdatafilesdatafilesdatafiles

UnifiedCatalogHMS/GlueHive/Hudi/Iceberg/DeltaLake?Hive/Hudi/Iceberg/DeltaLakeCatalogCatalogHMS/GlueUnifiedCatalo

网址：StarRocks+极速统一的湖仓分析.pptx http://mxgxt.com/news/view/1143843

相关内容

让数据分析极速统一！StarRocks和阿里云一起干了件大事
如何利用 StarRocks 加速 Iceberg 数据湖的查询效率
StarRocks和帆软软件达成战略合作，助力企业开启“极速统一”数字分析
StarRocks：从概念到应用的下一代分析型数据库
跨越速运张杰：基于 StarRocks 提升运单分析时效｜爱分析活动
StarRocks Summit 2023 技术交流峰会圆满落幕
StarRocks Summit Asia 2024落幕，Lakehouse引领数据技术新趋势
StarRocks 2024 数据技术峰会圆满收官，Lakehouse引领数据技术新趋势
StarRocks数据集成
StarRocks 2.1 新版本特性介绍

随便看看