Handling web pages with dynamic content

Last edited on

This example demonstrates how to handle pages with dynamic content. For this it uses Google's search engine page.

Keep in mind that Google's search engine page is highly dynamic in the sense that you must expect it to change rapidly, possibly even several times per day. Therefore this example simply demonstrates how to cope with the status quo at the time of writing this example.

Your own web pages will be very unlikely to change as often and as dramatically (internally) as Google's page, therefore it should be much more "stable" for you.

Python:

def main():
    startBrowser("www.google.com")
    snooze(3)

    # Getting Google's input field is not easy, so just
    # type blindly, and type a dummy value first, too:
    nativeType("x")
    snooze(2)
    nativeType("<Ctrl+a>")
    nativeType("froglogic squish")
    nativeType("<Return>")
    snooze(3)

    test.log("Fetch the results...")
    fetch_results()

    test.log("Go to next result page...")
    clickLink(waitForObject({"id": "pnnext", "tagName": "A"}))
    snooze(3)

    test.log("Fetch the results...")
    fetch_results()

def fetch_results():
    occurrence = 0
    result_elements = []
    result_dom_path_length = None
    while True:
        occurrence +=1
        n = {"tagName": "H3", "className": "r", "occurrence": occurrence}
        try:
            o = waitForObject(n, 6000)
        except:
            break

        # The first element has the desired DOM path length;
        # this is required to skip the sub-entries shown for
        # the first hit at Google:
        if result_dom_path_length is None:
            result_dom_path_length = len(o.domPath.split("."))
            result_elements.append(o)
            continue
        if len(o.domPath.split(".")) != result_dom_path_length:
            test.log("Ignoring sub-hit: %s" % o.innerText)
            continue
        result_elements.append(o)

    for i, o in enumerate(result_elements):
        test.log("#%s: %s" % (i+1, o.innerText))
test.py

Note how the solution makes assumptions along the way: