nach oben

Datenbank-Spektrum

Erschienen in:

03.02.2023 | Schwerpunktbeitrag

TPC, Where Art Thou?

verfasst von: Meikel Poess

Erschienen in: Datenbank-Spektrum | Ausgabe 3/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Performance has and still is perceived by customers as one of the main differentiators of computer systems. Hence, having access to widely accepted, objective and verifiable performance data is paramount for customers in making informed buying decisions. The last ten years have seen the rise of many new DBMSs. With them came a flurry of benchmark definitions. Even if technically sound, these benchmarks alone cannot achieve the goal of delivering widely accepted, objective and verifiable performance data. In addition they need to be agreed upon by a large body of vendors, and they need an independent oversight authority that assures correctness of their results. There are two major standard organizations that work on achieving these goals: Standard Performance Evaluation Corporation (SPEC) and Transaction Processing Performance Council (TPC). This article centers around the TPC, its history, challenges and steps it is taking to stay relevant in the years ahead.

Vorheriger Artikel Towards a Benchmark for Shared Databases [Vision Paper]

Nächster Artikel Comprehensive and Efficient Workload Summarization

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Datenbank-Spektrum

Datenbank-Spektrum ist das offizielle Organ der Fachgruppe Datenbanken und Information Retrieval der Gesellschaft für Informatik (GI) e.V. Die Zeitschrift widmet sich den Themen Datenbanken, Datenbankanwendungen und Information Retrieval.

Jetzt informieren

A DBMS Solution includes both hardware (central computer, network and any peripheral hardware, etc.) and software (Operating System (OS), DBMS, Transaction Processing (TP) monitor middleware, etc. A DBMS Solution vendor provides both hardware and software – in some cases partnering with software vendors.

source: Hive on Qubole runs 4x faster than Hive on Alternative Platforms https://www.qubole.com/blog/hive-on-qubole-runs-4x-faster-than-alternatives.

source:AtScale Cloud Data Warehouse Benchmark: Amazon Redshift https://www.atscale.com/wp-content/uploads/2019/11/ATS3346_Benchmark_Survey_Report_Redshift-updated_compressed.pdf.

source: TPC‑C Benchmark: 10,000 Warehouses on YugabyteDB https://www.yugabyte.com/blog/tpc-c-benchmark-10000-warehouses-on-yugabytedb.

source: Querying 6.35 Billion Records – a TPC-DS Performance and Cost Comparison between Big Data platforms Starburst Enterprise and EMR SQL engines https://www.concurrencylabs.com/blog/starburst-enterprise-vs-aws-emr-sql.

https://www.tpc.org/tpcds/results/tpcds_results5.asp?version=3.

https://docs.snowflake.com/en/user-guide/sample-data-tpcds.html.

https://docs.snowflake.com/en/user-guide/sample-data-tpch.html.

https://www.cockroachlabs.com/docs/v22.1/performance-benchmarking-with-tpcc-large.

https://docs.aws.amazon.com/athena/latest/ug/athena-prebuilt-data-connectors-tpcds.html.

Atomicity, Consistency, Isolation and Durability.

ResultID=121051901.

Heraclitus.

Hogan T (2009) Overview of TPC benchmark E: the next generation of OLTP benchmarks. In: Nambiar RO, Poess M (eds) Performance Evaluation and Benchmarking, first TPC technology conference TPCTC 2009, Lyon, France, August 24–28, 2009. Lecture Notes in Computer Science, vol 5895. Springer, Berlin, Heidelberg, pp 84–98. https://doi.org/10.1007/978-3-642-10424-4_7 (Revised Selected Papers)CrossRef

Nambiar RO, Poess M (2006) The making of TPC-DS. In: Dayal U, Whang K, Lomet DB, Alonso G, Lohman GM, Kersten ML, Cha SK, Kim Y (eds) Proceedings of the 32nd international conference on very large data bases Seoul, Korea, September 12–15, 2006 ACM, pp 1049–1058. http://dl.acm.org/citation.cfm?id=1164217

TPC-DS Benchmark Specification. https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-DS_v3.2.0.pdf. Accessed 7 Nov 2022

TPC-DS Benchmark Description. https://www.tpc.org/tpcds/default5.asp. Accessed 7 Nov 2022

Huppler K (2010) Price and the TPC. In: Nambiar RO, Poess M (eds) Performance evaluation, measurement and characterization of complex systems – second TPC technology conference TPCTC 2010, Singapore, September 13–17, 2010. Lecture Notes in Computer Science, vol 6417. Springer, Berlin, Heidelberg, pp 73–84. https://doi.org/10.1007/978-3-642-18206-8_6 (Revised Selected Papers)CrossRef

TPC-Energy Benchmark Specification. https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-Energy_v1.5.0.pdf. Accessed 12 Nov 2022

TPC-Pricing Benchmark Description. https://www.tpc.org/pricing/default5.asp. Accessed 2 Sep 2022

TPC-Pricing Benchmark Specification. https://www.tpc.org/TPC_Documents_Current_Versions/pdf/Pricing_v2.8.0.pdf. Accessed 22 Sep 2022

TPC-Energy Benchmark Description. https://www.tpc.org/energy/default5.asp. Accessed 7 Nov 2022

10.

Huppler K, Johnson D (2013) TPC express – a new path for TPC benchmarks. In: Nambiar R, Poess M (eds) Performance characterization and benchmarking - 5th TPC technology conference TPCTC 2013, Trento, Italy, 26 Aug 2013. Lecture Notes in Computer Science, vol 8391. Springer, Cham, pp 48–60. https://doi.org/10.1007/978-3-319-04936-6_4 (Revised Selected Papers)CrossRef

11.

Nambiar RO, Poess M, Dey A, Cao P, Magdon-Ismail T, Ren DQ, Bond A (2014) Introducing tpcx-hs: The first industry standard for benchmarking big data systems. In: Nambiar R, Poess M (eds) Performance characterization and benchmarking. Traditional to big data – 6th TPC technology conference TPCTC 2014, Hangzhou, China, September 1–5, 2014. Lecture Notes in Computer Science, vol 8904. Springer, Cham, pp 1–12 https://doi.org/10.1007/978-3-319-15350-6_1 (Revised Selected Papers)CrossRef

12.

Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen H (2013) Bigbench: towards an industry standard benchmark for big data analytics. In: Ross KA, Srivastava D, Papadias D (eds) Proceedings of the ACM SIGMOD international conference on management of data SIGMOD 2013, New York, NY, USA, June 22–27, 2013 ACM, pp 1197–1208 https://doi.org/10.1145/2463676.2463712CrossRef

13.

Bond A, Johnson D, Kopczynski G, Taheri HR (2015) Profiling the performance of virtualized databases with the tpcx‑v benchmark. In: Nambiar R, Poess M (eds) Performance evaluation and benchmarking: traditional to big data to Internet of things – 7th TPC technology conference TPCTC 2015, Kohala Coast, HI, USA, August 31 – September 4, 2015. Lecture Notes in Computer Science, vol 9508. Springer, Cham, pp 156–172. https://doi.org/10.1007/978-3-319-31409-9_10 (Revised Selected Papers)CrossRef

14.

Taheri HR, Little G, Desai B, Bond A, Johnson D, Kopczynski G (2018) Characterizing the performance and resilience of HCI clusters with the tpcx-hci benchmark. In: Nambiar R, Poess M (eds) Performance Evaluation and Benchmarking for the Era of Artificial Intelligence – 10th TPC Technology Conference TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018. Lecture Notes in Computer Science, vol 11135. Springer, Cham, pp 58–70. https://doi.org/10.1007/978-3-030-11404-6_5 (Revised Selected Papers)CrossRef

15.

Poess M, Nambiar R, Kulkarni K, Narasimhadevara C, Rabl T, Jacobsen H (2018) Analysis of tpcx-iot: the first industry standard benchmark for iot gateway systems. In: 34th IEEE international conference on data engineering ICDE 2018, Paris, France, April 16–19, 2018. ICDE, vol 2018. IEEE Computer Society, pp 1519–1530. https://doi.org/10.1109/ICDE.2018.00170CrossRef

16.

Cooper BF, Silberstein A, Tam E, Ramakrishnan R, Sears R (2010) Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM symposium on cloud computing SoCC 2010, Indianapolis, Indiana, USA, June 10–11, 2010, pp 143–154 https://doi.org/10.1145/1807128.1807152

17.

TPC Policies 6.18. https://www.tpc.org/tpc_documents_current_versions/pdf/policies_v6.18.0.pdf. Accessed 23 Sep 2022

Titel: TPC, Where Art Thou?
verfasst von: Meikel Poess
Publikationsdatum: 03.02.2023
Verlag: Springer Berlin Heidelberg
Erschienen in: Datenbank-Spektrum / Ausgabe 3/2022
Print ISSN: 1618-2162
Elektronische ISSN: 1610-1995
DOI: https://doi.org/10.1007/s13222-022-00428-9

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Datenbank-Spektrum

Weitere Artikel der Ausgabe 3/2022

Reviving the Workshop Series on Testing Database Systems – DBTest

The Big Data Analytics Group at Saarland University

Towards a Benchmark for Shared Databases [Vision Paper]

News

Benchmarking JSON Document Stores in Practice

Workshop on „Reproducible Science in Data Management“

Premium Partner