Open Telekom Cloud for Business Customers

DLI offers new features and new functionalities

Following features and functionalities are available.

New functionalities:



Load-based elastic resource pool, queue, and job scaling

  • SQL jobs can preempt resources based on loads (graceful scaling).
  • SQL queues can preempt resources based on loads.
  • In a SQL queue, the number of CUs applied for by the pod in the QuotaManager information can be customized.

Automatic scaling of general-purpose queues based on job priorities and resource loads

  • The job waiting list of the general queue is sorted by time and priority.
  • The ClusterAgent component of the cluster can collect backpressure metrics of Flink jobs.
  • The ClusterAgent can collect the number of Spark executors.

Permissions on DLI database tables can be granted by tag

  • Permissions on DLI database tables can be granted and authenticated by tag.
  • APIs that support granting permissions by tag are added.
  • On the DLI console, permissions on DLI tables can be granted by tag.

Fließende Skalierung von elastischen Ressourcenpools

  • Elastische DLI-Ressourcenpools können problemlos skaliert werden.
  • Allgemeine Warteschlangen können in elastische Ressourcenpools migriert werden.
  • Verschiedene Flavors in elastischen Ressourcenpools können zugewiesen und freigegeben werden.

DLI can connect to multiple catalogs

  • dli-catalog-client connects to DLI catalogs using independent configuration items.
  • Spark images connect to DLI catalogs.
  • The Spark Extension plugin connects to DLI catalogs using new configuration items.

New features:

  • On the Spark job management page, jobs can be filtered by date.
  • DLI permissions can be granted across tenants.
  • The Spark tenant plane can be interconnected with Quick BI through Livy.
  • Spark and Scala versions can be specified when interactive jobs are submitted using Livy.
  • DLI catalog metadata can be accessed through Spark Jar jobs.
  • The OLC feature allows DLI JobManager to dynamically control the flow at the microservice level.
  • DLI tenant clusters can be quickly switched with just one click. In some scenarios, jobs in existing clusters can be smoothly migrated to new clusters.
  • DLI SDKs and catalog clients can use temporary AKs/SKs and security tokens for authentication.
  • Flink1.12 OpenSource SQL jobs support datasource authentication.
  • DLI Spark jobs support 3.1*.
  • DLI overload detection can identify overload sources and top subscribers and APIs that cause overload.
  • DLI Catalog Client supports AK/SK authentication.
  • Pods in CCE clusters support balanced job scheduling.
  • DLI Spark datasource jobs support Spark 3.1*.

Further information can be found in Data Lake Insight HelpCenter.

*By default Spark 3.1 is disabled and customers need to open a support ticket to get enabled this version in their tenant.


Do you have questions?

We answer your questions about testing, booking and use – free of charge and individually. Try it! 
Hotline: 24 hours a day, 7 days a week
0800 3304477 from Germany / 00800 33044770 from abroad

  • Communities

    The Open Telekom Cloud Community

    This is where users, developers and product owners meet to help each other, share knowledge and discuss.

    Discover now

  • Telefon

    Free expert hotline

    Our certified cloud experts provide you with personal service free of charge.

     0800 3304477 (from Germany)

    +800 33044770 (from abroad)

    24 hours a day, seven days a week

  • E-Mail

    Our customer service is available free of charge via E-Mail

    Write an E-Mail