Learn about Steaming Data at Tech Coffee No.45

Tech Coffee #45 with the “Streaming Data in Azure Databricks” topic will share a real case from the project with National Cash Register (NCR) – a company specializing in providing automated teller machines (ATM). This is a project to process IoT Data from NCR devices, using Azure Services including Eventhub, Azure Table, Delta Lake, Databricks, ADX , ADF, etc.
Streaming data is generated continuously by thousands of different data sources. For streaming data, one of the biggest challenges is processing large amounts of data continuously in a short time. In order for the system to achieve fast and stable processing speed, Data Engineers need to optimize the data processing, ensuring the smallest delay in collecting and transporting data from the source.
Tech Coffee issue 44 will cover techniques used in setting up a streaming pipeline using Azure Databricks, streaming data from Delta Lake, Eventhub. The results helped the system increase processing speed from 4K message/s up to 20K message/s, reducing system latency from 12 hours to less than 1 hour.

The program has the participation of Mr. Hoang Ha (HaH5) – a Data Engineer from FHM.AVI unit. Mr. Ha has 5 years of experience in the software industry, including 2 years working in Data service on-premises and Cloud with jobs such as ETL, Streaming, Data Warehouse, Visualization and Microsoft Azure, Spark platforms, Skywise.
In addition to the content about best practices in Streaming data problems, Mr. Ha also shared some more tips & tricks in establishing connections and processing data on Azure Databricks, typically Azure Blob Storage, Azure Table API, SQL Server, etc.
The program takes place at 2 pm on July 28, on the platform of Microsoft Teams. Interested FSOFTers can register here from now until 3 pm on July 27. In particular, the organizer will give FSOFTers in Ho Chi Minh City preferential vouchers for food services at Trung Nguyen E-Coffee (F-Town 3) when they register and participate in the program.
For any questions, please contact [email protected] or Workplace Tech Coffee.