This article is dedicated to the subject of data grids, also called data tables, or datagrids, or datatables… Basically, the concept is simple: display your data as a table where each column corresponds to a data type. For example, Last name, First name, Address, Email…
However, having a good data grid that works on the web poses a number of technological challenges, especially when there is a lot of data to display.
The prerequisites of a good data grid
To be able to handle the data under good conditions, a good data grid should be able to:
- display or hide certain columns
- adjust column size
- change the order of the columns
- edit the content of the data grid easily (in place editing)
- sort data on one or more columns (sorting)
- filter the data to see only a sub-part (filtering)
- group data easily to perform consolidations (grouping)
- scroll through the pages quickly (paging)
- navigate very quickly in the data (virtual scrolling)
- store one or more configurations of the data grid, in order to quickly display several different views of the same information (multi-view)
Surprisingly, although these prerequisites seem very important, there are very few ready-to-use Vanilla JavaScript web components that meet all these conditions: most of the opensource components that we found are incomplete, buggy (memory leaks!), or depend on other javascript libraries (JQuery, React, Angular), and commercial components are quite expensive (sometimes with royalties), or are part of a complete framework (ExtJS, Webix, Telerik…).
In the end, nothing was really satisfactory, so we chose to start from the blank page, with the following objectives:
- simplest html markup (dumb)
- minimal html markup (performance)
- limitation of the number of nodes in the DOM (performance)
- limitation of the number of listeners in the DOM (performance)
- limitation of the number of CSS classes used (performance)
- aesthetic easy to modify (light, dark themes, etc …)
- easy to read code (dumb)
Alpha 0.1
Our alpha 0.1 version showed good results very quickly. On this first video, we see that we can already manage several tables simultaneously (multi-view) like spreadsheets. You can also see the following features:
- more than 50,000 rows per table without any difficulty
- column selection
- move columns with drag & drop
- multi-column sorting
- simple data filtering
- filters with complex logic: nesting AND / OR operators + drag & drop of filters
Alpha 0.2
Alpha 0.2 then had to integrate virtual scrolling. As a reminder, virtual scrolling requires displaying a “false” scrollbar allowing you to navigate in all of the data (here, 50,000 rows) while building only the rows that are visible on the screen (i.e. barely thirty)
In the end, our virtual scrolling shows performance that seems to be the best on the market compared to all the tables we had tested before, all without any memory leak during the construction and destruction of the DOM nodes:
Alpha 0.3
The next challenge for alpha 0.3 was going to be the integration of a major and complex functionality: the possibility of dynamically grouping the data… while keeping the virtual scrolling!
Three weeks of work later and a really good grouping system, we celebrated this with a video showing the full capabilities of the data table in alpha 0.3 version:
Alpha 0.4
But barely presented alpha 0.3, we were already asked to add an option to show or hide the hierarchy of groups. Thing done in alpha 0.4!
Alpha 0.5
Alpha 0.5 included three new key features for the future of the framework:
- the ability to have columns displayed with a different rendering without loss of performance (in the example below, the columns in color)
- the ability to visualize each row of the table as a form, as in PickaForm (it’s only a very basic work-in-progress that is shown on the video)
- reactivity, which allows the UI and data to always be synchronized correctly. For the test, we open the same data 3 times, and we can see on the video that the data always remains well synchronized in all of the displayed UI components :
Results
When we started this development project, we were not sure that we had made the right choice, because we did not know if we would be able – within a reasonable time – to obtain a data grid as good as those existing on the market.
For example, the challenge of combining both virtual scrolling and a dynamic data grouping system, all with good performance, seemed quite complex.
With hindsight and after having succeeded in this bet, it seems to us today that starting again from the blank page was the best (if not the only) of the options. Our alpha 0.5 version already fulfills all the expected objectives, since this data grid:
- is super fast
- is reactive
- allows you to sort, filter, and group data in an intuitive and dynamic way
- has maybe the most efficient virtual scrolling on the market
- has virtual scrolling that works even when data is consolidated on multiple levels
- is 100% vanilla javascript code, is ultra simple (dumb code), clean, and well documented
- has no memory leak
- has a footprint of 79Kb (source), 37Kb minified, and only 9Kb Gzipped
- has a light, simple, clean and efficient html markup:
To sum up, we didn’t think everything would turn out so well, and that heralds a lot of potential for future versions of PickaForm.