For the majority of web scraping applications, the Selector Tool within Axiom.ai will suit your needs to select elements on a page that you want to extract. However, there are times when you may need more options to select elements, such as extending the functionality that we offer, or accessing elements on the page that may hidden behind an overlay.

Introduction

CSS (Cascading Style Sheets) are used to control the appearance and layout of HTML elements on a webpage. It allows the developers of the website to apply styles such as colors, fonts, spacing, and positioning, ensuring a consistent and visually appealing design. Most modern websites will have multiple stylesheets that are used to construct the style of a site - some larger websites may have tens of these files!

In order to target the HTML elements on the page, CSS uses a variety of selectors to target the element based on a few methods: using the element tag, an ID, a class (or multiple) or an attribute. You can use a mixture of all of these to create a CSS selector, we will dive into CSS specificity next as an important concept to taking advantage of combining selectors.

CSS Specificity

CSS specificity should be kept in mind when writing CSS selectors, this is a set of rules which determine which styles are applied to an element when combining multiple selectors that are targetting the same element. It assigns weight to selectors based on their type, prioritising more specific rules over less specific one - hence the name! The hierarchy works as follows, listed from highest specificity to lowest:

1. Inline styles such as style="color: blue;"
2. ID selectors such as #list-item
3. Class selectors, attributes, and pseudo-classes such as .list-item, [type="checkbox"], :hover
4. Element selectors and pseudo-elements such as div, p, ::before

When multiple rules conflict, the browser will use the rule with the highest specificity. If specificity is equal, the last declared rule will prevail.

Learning how to take advantage of specificity gives you the advantage of being able to avoid using the dreaded !important attribute which can have unexpected side effects when used where it's not needed.

Knowledge check: CSS specificity

In the following code, what color will the text be when the CSS is applied?

<p id="text-1" class="my-text">
    Hello, World!
</p>

.my-text { color: red; }

#text-1 { color: green; }

p { color: blue; }

Jump to answer

Combinators

Combinators are CSS elements that can be used to relate two CSS rules. These can be useful in addition to CSS specificity to ensure that you are targetting the right elements.

Child combinators (opens new window) use the > to match elements that are matched by the second selector that are direct children of the elements matched by the first. For example, div > span will match the following:

<div>
    <span>This is matched as a direct child of the div element.</div>
</div>
<span>This is not matched as it is not a direct child of a div element.</span>

Descendant combinators (opens new window) use a single space character to combine two selectors such that elements matched by the second selector are selected if they have an ancestor element matching the first. For example, div span will match the following:

<div>
    <span>This is matched as a child of the div element.</div>
</div>
<span>This is not matched as it is not a child of a div element.</span>

There are other combinators that can be used to enhance your custom selectors, but they should suffice to get you started.

Identifying selectors to use

In order to use selectors, you'll first need to investigate which selectors will work for your use case. Generally, assuming you're not the developer of the site, Chrome Devtools (opens new window) is the best tool to use to inspect the underlying code of the page. To open Devtools, right-click on the page and click "Inspect" in the context menu that appears. It should automatically drop you in the "Elements" tab, but if it doesn't, navigate there.

We recommend having some web development knowledge from this point, however, you do not need to fully understand the code that is displayed in order to extract a CSS selector provided you have read the sections above.

To jump to the code behind an element, right-click on the element on the page and click "Inspect" again within the context menu that appears. Provided it does not have an overlay, the window should jump to the section of the code that contains the element where you will be able to extract a selector. For a very specific selector, you can right-click on the element in the code, click "copy" and then "copy selector" - we wouldn't really recommend this method as some sites use frameworks that mean that the ID may change frequently and this would break your automation if the site even has a small update.

Using CSS in your steps

Quite a few Axiom.ai steps allow for custom CSS selectors to be used to target elements on the page as your automation runs, primarily the Scrape and Interact steps. For example, to use a custom CSS selector within the Click element step, perform the following steps:

1. Search for and add a "Click element" step to your automation.
2. Click "Select".
3. Click "Custom".
4. Enter your custom selector.

Dynamically setting a custom CSS selector

While explicitly setting the custom CSS selector within your steps is useful for consistently interacting with an element on a page, there may be times when you need to dynamically set this using a data token. To get started, click "Set selector from data" within the custom CSS selector input within the step, this will prompt you to select a data token from your automation.

There may even be times where you need to loop through a list of custom selectors. In order to loop through a list of CSS selectors, you will first need to create a list of selectors. This can be useful when you may need to click through a list of items. For example:

Selector
list-item-1
list-item-2
list-item-3

Tip: this may be stored in a Google Sheet for ease.

Once you have your list of custom CSS selectors, you can use the following pattern to loop through your list of selectors:

1. Add a Read data from a Google Sheet step, select the sheet that contains your list of selectors.
2. Add a Loop through data step, setting the google-sheet-data data token as the data to loop through.
3. Add a Scrape or Interact step, setting the google-sheet-data token inside the "Loop through data" step as the custom CSS selector.
4. Add any other steps inside the loop.

This can be useful in situations where you have a list of items that you wish to click, perform actions, and then continue on to click on the next item of the list. This can be used to perform multiple actions on lists, for checkboxes, for example.

Wrapping up

Learning how to take advantage of custom CSS selectors can be a powerful skill to have in your toolbox - whether this is for use within your Axiom.ai automations, or generally in any web development adjacent field. This can enhance your automations to ensure consistent results on pages that may change frequently, or are not suitable to the standard features of the Selector Tool.

Further reading

If you're interested in learning more about CSS and CSS selectors, check out the resources below:

• Mozilla Developer Docs (opens new window)
• W3Schools (opens new window)

Knowledge check answers

CSS Specificity

As we have a CSS selector using the ID (#text-1) this will be the highest specificity, followed by .my-text and finally the p selector.