Big Lists Targeting

Big Lists Targeting is a feature developed by BidSwitch to enable users to filter traffic based on very large sets of data. List contents are checked in real-time against bid request values to let you either include or exclude specific records within a Targeting Group. Lists can be used for filtering in two ways.

Within the real-time system, big lists are stored as an Xor filter which allows for an item to be checked if it is part of a set. Xor filters are a faster and more concise alternative to Bloom filters, but the lookup using an Xor filter is not 100% exact and can lead to false positives i.e. an item can be marked as present in a list, whereas in reality it isn’t. The false positive rate is about 0.3%. The following fields within bid requests can be targeting using big lists, and more will be added in the near future.

Big list Support Types

List Type

Description

UUIDs

Filters IDs based on the BidSwitch UUID in the user.id field for website traffic, see the User Object for more details, or the device.ifa field for in-app traffic, see the Device Object for more details.

Bundles

Filters based on the app.bundle field.

Domains

Filters based on the site.domain field.

IP Addresses

Filters based on device.ip and device.ipv6 fields. IP Addresses of both types can be present in the same list i.e. IPv4 and IPv6

List Restrictions

  • When referring to lists for traffic filtering, BidSwitch supports two types:

    • Normal lists uploaded through the User Interface and set as part of Targeting Groups, these lists have a maximum limit of 5,000 records.

    • Big lists uploaded to a dedicated Google Cloud Bucket, which are also set as part of Targeting Groups but which are not controlled or managed through the UI.

  • Big lists do not have any limit as such nor is there a limit on the number of lists you create, but your account is limited to 500MB of overall list records. For comparison, a list of 2 million UUIDs is approximately 74MB, so 6-7 such big lists is approximately your limit. Limit details may change as the feature sees more adoption.

  • Lists are not updated in real-time, when you make updates to a list it will take approx 20 minutes for those changes to take effect on real-time targeting.

  • To use the Big Lists feature requires setting up a Google service account and making making sure access is granted to the bucket for both you and BidSwitch so that the lists can be used and the data pipelines automated.

Workflow Overview

Big Lists Targeting requires the support team to enable it on your account and for the Ops team to setup a dedicated bucket. Contact your Account Manager for more details, but once set up the workflow is as follows.

  1. Upload your lists to the dedicated GCE bucket.

  2. Shortly after upload, your lists become available to use when configuring Targeting Groups. The list name in the UI/API matches the filename uploaded to the bucket, i.e. if you uploaded a file named app_retargeting_list.tsv, it will be labelled app_retargeting_list in the UI.

  3. You can configure as many Targeting Groups you wish to use each list. You may also use lists of different types per Targeting Group, e.g. you could target a list of UUIDs and exclude a list of app bundles.

  4. When a list is being used by Targeting Groups, you can make changes to it by uploading an updated list with the same file name to overwrite the existing contents in the bucket.

    • When updating a list to a newer version, it takes approx 20 minutes for the updated changes to take effect on real-time filtering.

    • If deleting a list, make sure it is not currently in use.

List Format

  • Each list must contain only one type of record, i.e. bundles, UUIDs, IP addresses, or domains.

  • Each list must be a plain text file with each item separated by a newline.

  • All lines are converted to lowercase and read without any modifications (i.e. all spaces, commas, and other symbols are considered part of string value). The only validation that is performed is to check that all lines contained in the file are valid UTF-8, no other checks are performed.

  • When configuring a Targeting Group, a list’s content is not validated against the targeting type, so if you upload a list of UUIDs but set that list to filter bundles, the resulting mismatch will not filter records as you expect.

Example UUIDs
f77a8e70-80da-4178-9ff4-4afaefd722b8
52cfb32e-16ae-4688-b7aa-6be553f099a0
0f7ffdc6-7ff1-42f0-add9-7fda3d2af924
Example Domains
example.com
site.co.uk
baguette.fr
Example Bundles
com.tripledot.woodoku
com.pixel.art.coloring.color.number
591560124
Example IPv4 Addresses
1.1.1.1
127.0.0.1
8.8.4.4.
Example IPv6 Addresses
2001:db8:3333:4444:5555:6666:7777:8888
2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF
2345:0425:2CA1:0000:0000:0567:5673:23b5

Targeting Group Configuration

Assuming you have both Big List targeting enabled on your account and you have uploaded lists to your dedicated bucket, you can apply those lists within Targeting Groups to filter your traffic.

  1. From within the myBidSwitch UI, open the targeting group you wish to configure: Targeting Groups ‣ [Your tgroup] ‣ Platform Settings ‣ Big List Targeting

  2. Configure the list filtering details:

    • For the Big List Type setting, set the field to filter, e.g. UUIDS

    • For the Filtering Mode setting, set whether to include or exclude the list values from the traffic your receive

    • For the List Name setting, set the list to be used for this filter. Note: If you have not uploaded a list to your GCE bucket there will be no option on the drop down menu.

  3. Save the settings

Important

When configuring a Targeting Group, a list’s content is not validated against the list type you set, so you must ensure you select the correct list type, i.e. if you use a list of UUIDs to filtering app bundles, the resulting mismatch will not filter traffic as you expect.