Important Dates

Paper submission deadline: May 1, 2013

Paper acceptance notification: May 15, 2013

Workshop: July 16-17, 2013

Full-length paper submission deadline: August 16, 2013

 

Third Workshop on Big Data Benchmarking, July 16-17, 2013 Xi'an, China

The Third Workshop on Big Data Benchmarking (3rd WBDB) will be held on July 16-17, 2013 in Xi'an, China, hosted by the Shanxi HPC Center.

The objective of the WBDB workshops is to make progress towards developing industry standard benchmarks for evaluating hardware and software systems for big data applications.

The BigData Top100 List

The BigData Top100 List concept emerged from discussions at previous WBDB workshops and related meetings. The list would rank big data systems according to their performance on selected big data analytics workloads, enabling comparisons among different big data solutions.

A successful benchmark would be simple to implement and execute; cost effective, so that the benefits of executing the benchmark justify its expense;timely, with benchmark versions keeping pace with rapid changes in the marketplace; and verifiable so that results of the benchmark can be validated via independent means.

Workshop Themes

The 3rd WBDB will emphasize two benchmark proposals that are currently being considered: one based on a Deep Analytics Pipeline for event processing[1] and a second based on extending the TPC-DS benchmark with semistructured and unstructured data and new queries targeted at those data, called BigBench[2]. The priority is to address the following issues in the context of those benchmark proposals:

  • Data generation: Models and procedures for generating synthetic data with requisite properties.
  • Workload: Representative big data business problems and corresponding specific implementations foreach step and/or query in the workload.
  • Benchmark execution: Rules and regulations for running the benchmark; data scale factors; benchmark versioning; benchmark metrics.
  • Metrics for efficiency: Measuring the efficiency of the solution, e.g. based on costs of acquisition, ownership, energy and/or other factors.

Papers on early implementations of the Deep Analytics Pipeline or BigBench, or describing lessons learned in benchmarking big data applications are solicited. Discussions of enhancements to these benchmarks are also encouraged, for example, including more data genres (e.g. graphs) in the workload; considering a range of machine learning and other algorithms, etc. Papers proposing other benchmarking alternatives will also be considered.


  • Presentation on Deep Analytics Pipeline [ audio ]. Enter your name.

  • BigBench: Towards an Industry Standard Benchmark for Big Data Analytics, to appear in SIGMOD Conference 2013

Workshop Program

  • The workshop will include invited talks, regular presentations, “lightning” talks, and discussion sessions.
  • A registration fee of US$150 will be charged to cover workshop expenses.
  • Attendance will be capped to ensure effective discussions and active participation.
  • In selecting papers, preference will be given to papers that directly address the themes of the workshop and to diversity of representation across organizations and institutions.
  • Extended versions of selected papers will be published in the Springer Verlag Lecture Notes in Computer Science series.
  • Workshop sponsors will be provided a speaking slot. Sponsorship opportunities are available. Please contact Chaitan Baru, baru@sdsc.edu if you are interested.

Paper Formatting Instructions

Papers should be formatted using the Springer LNCS Proceedings format. Selected revised papers will be published in Lecture Notes in Computer Science by Springer.



Important Dates (updated)

  • May 15, 2013: Short versions of papers (4-8 pages) should be submitted by May 1, 2013, using the CMT system.
  • May 29, 2013: Authors will be notified about paper acceptance.
  • August 16, 2013: Full versions (16 pages) of accepted papers should be submitted by (after the workshop),

Submission should be via the CMT conference management service:

https://cmt.research.microsoft.com/WBDB2013/

The meeting is supported by sponsorship from Greenplum, Mellanox, Brocade, NetApp, Seagate, and in-kind support from the Shanxi HPC Center, XI'an, China.

3rd WBDB Call for Papers [ pdf ]