New to Telerik UI for WinForms? Download free 30-day trial

High Performance with RadGridView and Virtual Mode including Filtering, Sorting and Grouping

Date Posted Product Author
Q1 2013 RadGridView Georgi Georgiev

Problem

You want to bind RadGridView to over a million items. In such cases it is recommended to use Virtual Mode, however this removes the Filtering, Sorting and Grouping capabilities of RadGridView since RadGridView has no direct access to its DataSource.

As of Q1 2016 (version 2016.1.112) Telerik UI for WinForms suite offers RadVirtualGrid control. It is a grid component developed on top of Telerik Presentation Framework which provides a convenient way to implement your own data management operations and optimizes the performance when interacting with large amounts of data.

Solution

We can create a special Data Layer which will manage our DataSource and only provide the needed data to the grid. This layer will also allow RadGridView to be responsive while the operations are being executed. Below you can see the speed at which the operations will be performed when *RadGridView *is bound to 2 million items:

high-performance-with-radgridview

NOTE: The projects below requires NET 4 since it uses some NET 4 features, such as Parallel LINQ and UI for Winforms version *Q1 2014 or above** due to the paging functionality and some API changes.*

First of all we need to create our Data Layer. It will be called ItemSource. It needs a few essential properties:

  • DataSource – This will be the property which will keep the collection with all items which were initially passed to it.
  • View – This will be the collection of items which will be represented by RadGridView. This collection will always be sorted or filtered according to the descriptors of RadGridView.
  • BoundProperties – This collection will keep all the properties of the bound objects, so we can get or set the value when needed.
  • Count – Will return the amount of items in the View.
  • Indexer – Will return a value from the View by index.
  • CurrentOperation – This will allow us to track what operation is executing at any given moment

The implementation of the above properties can be seen below: 

public ItemSourceOperation CurrentOperation { get; private set; }

public object DataSource
{
    get
    {
        return this.dataSource;
    }
    set
    {
        if (value == null)
        {
            this.dataSource = this.view = null;
            return;
        }

        IList dsList = ListBindingHelper.GetList(value) as IList;
        if (dsList == null || this.dataSource == dsList)
        {
            return;
        }

        this.dataSource = this.view = dsList;
        this.boundProperties = null;
    }
}

public IList View
{
    get { return this.view; }
}

public object this[int index]
{
    get { return this.view[index]; }
}

public PropertyDescriptorCollection BoundProperties
{
    get
    {
        if (this.dataSource == null || this.dataSource.Count == 0)
        {
            return new PropertyDescriptorCollection(null);
        }

        if (boundProperties == null)
        {
            boundProperties = ListBindingHelper.GetListItemProperties(this.dataSource);
        }

        return boundProperties;
    }
}

public int Count
{
    get
    {
        return this.view.Count;
    }
}

It will also need to have methods which will allow us to get the values from the items in the DataSource, namely GetValue and SetValue:

public virtual bool SetValue(object entry, int index, object value)
{
    if (!this.BoundProperties[index].IsReadOnly)
    {
        this.BoundProperties[index].SetValue(entry, value);
        return true;
    }

    return false;
}

public virtual bool SetValue(object entry, string member, object value)
{
    if (!this.BoundProperties[member].IsReadOnly)
    {
        this.BoundProperties[member].SetValue(entry, value);
        return true;
    }

    return false;
}

public virtual object GetValue(object entry, int index)
{
    return this.BoundProperties[index].GetValue(entry);
}

public virtual object GetValue(object entry, string member)
{
    return this.BoundProperties[member].GetValue(entry);
}

Now, our ItemSource has to be able to perform the very important Sorting and Filtering operations. For the sorting we will use PLINQ. The queries will be executed over the view. Here is how our sorting method will look like:

protected virtual void Sort(SortDescriptorCollection descriptors)
{
    SortDescriptor[] currentDescriptors = descriptors.ToArray();
    if (this.CurrentOperation != ItemSourceOperation.None)
    {
        return;
    }

    if (currentDescriptors.Length == 0)
    {
        this.view = this.dataSource;
        this.Filter(this.masterTemplate.FilterDescriptors);

        return;
    }

    this.CurrentOperation = ItemSourceOperation.Sorting;
    List<object> sortView = this.view as List<object>;

    if (sortView == null)
    {
        sortView = new List<object>(this.view.Count);
        foreach (object item in this.view)
        {
            sortView.Add(item);
        }
    }

    ParallelQuery<object> query = sortView.AsParallel();
    SortDescriptor firstDescriptor = currentDescriptors.First();
    if (firstDescriptor.Direction == ListSortDirection.Descending)
    {
        query = query.OrderByDescending(x => this.GetValue(x, firstDescriptor.PropertyName));
    }
    else
    {
        query = query.OrderBy(x => this.GetValue(x, firstDescriptor.PropertyName));
    }

    OrderedParallelQuery<object> orderedQuery = query as OrderedParallelQuery<object>;
    for (int i = 1; i < currentDescriptors.Length; i++)
    {
        SortDescriptor currentDescriptor = currentDescriptors[i];
        if (currentDescriptor.Direction == ListSortDirection.Descending)
        {
            orderedQuery = orderedQuery.ThenByDescending(x => this.GetValue(x, currentDescriptor.PropertyName));
        }
        else
        {
            orderedQuery = orderedQuery.ThenBy(x => this.GetValue(x, currentDescriptor.PropertyName));
        }
    }

    this.view = orderedQuery.ToList();
    this.CurrentOperation = ItemSourceOperation.None;
}

For the filtering we will use the ExpressionParser which RadGridView uses internally to check whether an object passes certain filter:

protected virtual void Filter(FilterDescriptorCollection descriptors)
{
    FilterDescriptor[] currentDescriptors = descriptors.ToArray();
    ExpressionNode node = ExpressionParser.Parse(descriptors.Expression,this.masterTemplate.CaseSensitive);

    if (this.CurrentOperation != ItemSourceOperation.None || node == null || currentDescriptors.Length == 0)
    {
        this.view = this.dataSource;
        this.CurrentOperation = ItemSourceOperation.None;
        return;
    }

    this.CurrentOperation = ItemSourceOperation.Filtering;
    List<object> newView = new List<object>();
    IList filteredView = this.dataSource;

    for (int i = 0; i < filteredView.Count; i++)
    {
        if (!this.perform)
        {
            this.CurrentOperation = ItemSourceOperation.None;
            return;
        }

        object entry = filteredView[i];
        ExpressionContext context = new ExpressionContext();

        for (int j = 0; j < currentDescriptors.Length; j++)
        {
            string member = currentDescriptors[j].PropertyName;
            if (!context.ContainsKey(member))
            {
                context.Add(member, this.GetValue(entry, member));
            }
            else
            {
                context[member] = this.GetValue(entry, member);
            }
        }

        object evalResult = node.Eval(null, context);

        if (evalResult is bool && (bool)evalResult)
        {
            newView.Add(entry);
        }
    }

    this.view = newView;
    this.CurrentOperation = ItemSourceOperation.None;
}

The grouping will still be performed by RadGridView, due to its nature. That is why it is recommended to use the Paging functionality along with the PagingBeforeGrouping property set to true. Now we can filter or sort our data, however these methods are protected, moreover, even if we execute them they will be synchronous, which will block the UI thread. We will prevent that by implementing a queuing mechanism which will execute these methods and provide events for when the operations are complete. For this we will use a BackgroundWorker:

protected BackgroundWorker BackgroundWorker
{
    get { return this.backgroundWorker; }
}

The WorkerCompleted, ProgressChanged and DoWork events will be used to respectively:

  • Start another cycle which will execute tasks if there are such and report that there are no more tasks to execute

  • Report that a task is finished

  • Dequeue tasks and execute them in order

We will use one class and one enumeration that in the process of execution of pending tasks:

public enum ItemSourceOperation
{
    None = 0,
    Filtering = 1,
    Sorting = Filtering << 1
}

public class ItemSourceOperationEventArgs : EventArgs
{
    public ItemSourceOperation OperationType { get; private set; }

    public ItemSourceOperationEventArgs(ItemSourceOperation operation)
    {
        this.OperationType = operation;
    }
}

And these are the event handlers of the background worker, described above:

protected virtual void BgWorkerRunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
    if (this.queuedOperations.Count > 0)
    {
        this.perform = true;
        this.backgroundWorker.RunWorkerAsync();
    }

    this.OnOperationCompleted((ItemSourceOperationCompletedEventArgs)e.Result);
}

protected virtual void BgWorkerProgressChanged(object sender, ProgressChangedEventArgs e)
{
    this.OnOperationStarted(new ItemSourceOperationEventArgs((ItemSourceOperation)e.UserState));
}

protected virtual void BgWorkerDoWork(object sender, DoWorkEventArgs e)
{
    ItemSourceOperation operations = default(ItemSourceOperation);
    while (this.queuedOperations.Count > 0 && this.perform)
    {
        OperationParameters currentParams;
        if (!this.queuedOperations.TryDequeue(out currentParams))
        {
            System.Threading.Thread.Sleep(100);
            continue;
        }

        this.backgroundWorker.ReportProgress(0, currentParams.Operation);
        switch (currentParams.Operation)
        {
            case ItemSourceOperation.Filtering:
                this.Filter(currentParams.Descriptors as FilterDescriptorCollection);
                break;
            case ItemSourceOperation.Sorting:
                this.Sort(currentParams.Descriptors as SortDescriptorCollection);
                break;
            default:
                break;
        }

        operations |= currentParams.Operation;
    }

    e.Result = new ItemSourceOperationCompletedEventArgs(operations, !this.perform);
}

After we have a mechanism which can execute tasks in order we need a way to start it up. We will expose one method which be responsible for this:

public void PerformOperation(ItemSourceOperation operation, IList descriptors, bool force = false)
{
    switch (operation)
    {
        case ItemSourceOperation.Filtering:
            FilterDescriptorCollection filterDescriptors = descriptors as FilterDescriptorCollection;
            if (this.lastFilterExpression == filterDescriptors.Expression && !force)
            {
                return;
            }

            this.lastFilterExpression = filterDescriptors.Expression;
            break;
        case ItemSourceOperation.Sorting:
            SortDescriptorCollection sortDescriptors = descriptors as SortDescriptorCollection;
            if (this.lastSortExpression == sortDescriptors.Expression && !force)
            {
                return;
            }

            this.lastSortExpression = sortDescriptors.Expression;
            break;
        default:
            break;
    }

    OperationParameters operationParams = new OperationParameters(operation, descriptors);
    this.queuedOperations.Enqueue(operationParams);
    if (this.backgroundWorker.IsBusy)
    {
        OperationParameters peek;
        bool peeked = this.queuedOperations.TryPeek(out peek);
        if (peeked && peek.Operation == operationParams.Operation && peek != operationParams)
        {
            OperationParameters dequeued;
            this.queuedOperations.TryDequeue(out dequeued);
        }
        else if (peeked && peek.Operation != operationParams.Operation)
        {
            this.perform = false;
        }
    }

    if (!this.backgroundWorker.IsBusy)
    {
        this.perform = true;
        this.backgroundWorker.RunWorkerAsync();
        this.OnOperationStarted(new ItemSourceOperationEventArgs(operationParams.Operation));
    }
}

Now, we need to integrate this layer into RadGridView. To do this we will inherit from RadGridView and add a few properties:

  • ItemsSource – This will keep a reference to the ItemSource which will manage the data

  • VirtualDataSource – It will set the DataSource of the ItemSource and initialize the Rows and Columns according to the properties and the amount of items

  • ShowLoadingOverlay – Determines whether a waiting bar will be displayed in the middle of RadGridView while the operations are being performed in the background

  • AutomaticallyRetreiveCellValues – As we know, in VirtualMode we need to provide the values of the cells by ourselves. In this case this is not a trivial task, since many factors such as groups and pages should be taken into consideration, that is why we will provide a built-in way to do this.

  • AutomaticallyPushCellValues – The same as the previous property, however this time we will set the value of the data bound items.

  • LoadingOverlay – Provides reference to the waiting bar being displayed in the middle of RadGridView 

public ItemSource ItemsSource
{
    get { return (this.GridViewElement as VirtualRadGridViewElement).ItemsSource; }
}

public object VirtualDataSource
{
    get { return this.ItemsSource.DataSource; }
    set
    {
        this.FilterDescriptors.Clear();
        this.SortDescriptors.Clear();
        this.GroupDescriptors.Clear();
        this.ItemsSource.DataSource = value;
        this.Initialize();
    }
}

public bool ShowLoadingOverlay { get; set; }

public bool AutomaticallyRetreiveCellValues { get; set; }

public bool AutomaticallyPushCellValues { get; set; }

public RadWaitingBar LoadingOverlay
{
    get { return this.loadingOverlay;
}

The Initialize method is very essential as it manages the rows and columns of RadGridView. It should be executed either when the DataSource changes or when BindingContext changes:

protected override void OnBindingContextChanged(EventArgs e)
{
    base.OnBindingContextChanged(e);
    this.Initialize();
}

protected virtual void Initialize()
{
    if (this.BindingContext == null)
    {
        return;
    }

    if (this.RowCount == this.ItemsSource.Count && this.ColumnCount == this.ItemsSource.BoundProperties.Count)
    {
        return;
    }

    this.InitializeRowsAndColumns();
}

protected virtual void InitializeRowsAndColumns()
{
    if (!this.IsLoaded || this.VirtualDataSource == null)
    {
        return;
    }

    this.BeginUpdate();
    this.InitializeColumns();
    this.InitializeRows();
    this.EndUpdate();
}

protected virtual void InitializeColumns()
{
    if (this.ColumnCount != this.ItemsSource.BoundProperties.Count)
    {
        this.ColumnCount = this.ItemsSource.BoundProperties.Count;
        for (int i = 0; i < this.Columns.Count; i++)
        {
            PropertyDescriptor prop = this.ItemsSource.BoundProperties[i];
            GridViewDataColumn newColumn = GridViewHelper.AutoGenerateGridColumn(prop.PropertyType, null);
            newColumn.HeaderText = prop.DisplayName;
            newColumn.Name = prop.Name;

            this.Columns.RemoveAt(i);
            this.Columns.Insert(i, newColumn);
        }
    }
}

protected virtual void InitializeRows()
{
    if (this.RowCount != this.ItemsSource.Count)
    {
        this.RowCount = this.ItemsSource.Count;
    }
}

Now we just need to replace some of the classes of RadGridView. First we start with the RadGridViewElement:

protected override RadGridViewElement CreateGridViewElement()
{
    return new VirtualRadGridViewElement();
}

In the VirtualRadGridViewElement we will need to create a custom MasterTemplate:

protected override MasterGridViewTemplate CreateTemplate()
{
    return new VirtualMasterGridViewTemplate();
}

And in the VirtualMasterGridViewTemplate, a custom ListSource:

protected override GridViewListSource CreateListSource()
{
    return new VirtualGridViewListSource(this);
}

In the ListSource we need to set the DataBoundItem to our rows manually, in order to be able to group by these rows and create a custom RadCollectionView:

protected override void InsertItem(int index, GridViewRowInfo item)
{
    base.InsertItem(index, item);
    this.InitializeBoundRow(item, this.DataView.ItemsSource[index]);
}

protected override Telerik.WinControls.Data.RadCollectionView<GridViewRowInfo> CreateDefaultCollectionView()
{
    this.DataView = new VirtualGridDataView(this);
    return this.DataView;
}

The DataView actually creates the ItemSource, the GroupBuilder and the Indexer which simply keeps reference to the rows:

public VirtualGridDataView(GridViewListSource listSource)
    : base(listSource)
{
    this.listSource = listSource;
}

public GridViewListSource ListSource
{
    get { return this.listSource; }
}

protected override Telerik.Collections.Generic.Index<GridViewRowInfo> CreateIndex()
{
    return new VirtualIndex(this);
}

protected override GroupBuilder<GridViewRowInfo> CreateGroupBuilder()
{
    return new VirtualGroupBuilder(this.Indexer, this);
}

This is what the VirtualIndex requires to work properly:

public VirtualIndex(RadCollectionView<GridViewRowInfo> collectionView)
    : base(collectionView)
{
}

public override IList<GridViewRowInfo> Items
{
    get
    {
        return (this.CollectionView as VirtualGridDataView).ListSource;
    }
}

protected override void Perform()
{
}

And this is what our GroupBuilder requires:

private VirtualGridDataView dataView;

public VirtualGroupBuilder(Telerik.Collections.Generic.Index<GridViewRowInfo> index, VirtualGridDataView dataView)
    : base(index)
{
    this.dataView = dataView;
}

protected override object GetItemKey(GridViewRowInfo item, SortDescriptor descriptor)
{
    return this.dataView.ItemsSource.GetValue(item.DataBoundItem, descriptor.PropertyIndex);
}

NOTE: In order to see the full implementation of the classes you should download the source code below.

In order to use the grid it is enough to set the VirtualDataSource property to some collection. You can also use the OperationStarted and OperationCompleted events at will.

A complete solution in C# and VB.NET can be found here.

In this article
Related articles
Not finding the help you need? Improve this article