Article originally posted on InfoQ. Visit InfoQ

Introduction

Imagine you are scrolling through recent news or social media and suddenly everything slows down. Images load pretty slowly, videos refuse to play, and that one particular article that you were most interested in reading is taking forever to load. A lot of times, this happens because of slow or intermittent internet connection. Apps and websites have traditionally displayed content by using pagination, which involves loading content in pages or chunks. It is similar to reading a book where you only get to view one or two pages at a time.

This is effective, however this approach normally doesn’t live up to user expectations, especially on mobile devices with poor network connectivity. Issues such as slow loading, disruptive page transitions, and endless clicking or tapping for more content can annoy and ultimately drive users away from the service.

Whether it is online shopping, browsing social media, or catching up on news, users expect to have a seamless and engaging experience irrespective of device or network conditions. This is where we could leverage AI for smart and personalized pagination.

Instead of providing a single static pagination approach for all the users, we could leverage AI to power dynamic content loading. This dynamic pagination approach allows varying the presentation of the content to suit each individual user’s particular needs and network capabilities.

Consider a website or app that somehow is able to sense your internet connectivity conditions as well as device and adjusts what it shows accordingly. With a high speed connection, you might see a rich multimedia intensive page. On slow connections, the same page would prioritize essential text and images first and then load other chunks as bandwidth allows.

By smartly adjusting the amount of content to be loaded, we can create a more responsive and personalized experience for every user. Users benefit because this approach enables faster loading times and smoother interactions.

Companies benefit in lowering technical infrastructure costs because this reduces the amount of data transferred thereby significantly cutting down the load on their streaming servers and network resources. Additionally, reduced latency means happier users; happier users are more likely to stay engaged and keep coming back.

In this article, we will examine how we could use AI to personalize content loading based on user behaviour.

Leveraging AI for Analyzing User Behavior

Consider two users on a news website:

• User A: Quickly scrolls through headlines and primarily clicks on articles with eye-catching images.
• User B: Read each article thoroughly and spend considerable time on every page.

These usage patterns can be identified by AI and the delivery of the content can be adjusted appropriately, such as prefetching images for User A or the next full article for User B. For an AI to effectively personalize content delivery, it must first understand the user’s distinct engagement patterns and infer their intent. This understanding is built by analyzing specific behavioral signals picked up during their interaction with the content. The subsequent sections will delve into practical methods for capturing and interpreting two crucial types of these signals:

• Scroll depth and speed tracking reveal how users navigate through pages and their pace.
• Dwell time analysis helps quantify users’ interest and engagement with particular sections or elements.

By examining these behaviors, the AI can build a more accurate profile to anticipate user needs.

Scroll Depth and Speed Tracking

Client Side Sample Implementation

Here, on the client side, we use JavaScript event listeners (such as scroll or wheel) to track how far down the page and how quickly a user scrolls. This data provides clues about their engagement and interest level.

The following example uses a scroll event listener:

window.addEventListener('scroll', () => { 
// Get the current scroll position 
const scrollTop = window.pageYOffset || document.documentElement.scrollTop; 
// Get the total scrollable height of the document. 
const scrollHeight = document.documentElement.scrollHeight - 
 document.documentElement.clientHeight; 
// Calculate the percentage of the page scrolled
const scrollPercentage = (scrollTop / scrollHeight) * 100; 
// Log the scroll position and percentage 
console.log('Scroll Top:', scrollTop); 
console.log('Scroll Percentage:', scrollPercentage); 
// You can also track the scroll speed here by comparing the current scroll 
// position with the previous one and calculating the difference over time. 
});

Here is an example of wheel event listener:

window.addEventListener('wheel', (event) => { 
// Get the deltaY value, which indicates the scroll direction and 
// amount.
const deltaY = event.deltaY; 
// Log the scroll direction and amount. 
console.log('Scroll Delta:', deltaY); 
// You can use this information to calculate scroll speed and acceleration more
// accurately. 
}); 

Server Side Sample Implementation

Once our server receives the scroll or wheel events from the client device, we can apply ML models such as regression or decision trees to analyze scroll depth and speed patterns to predict how much content a user is likely to consume. For example, if a user consistently scrolls rapidly to the bottom of the page, the system can anticipate their need for more content and proactively load the next chunk.

Regression model example

Let’s assume we are tracking the following data points for each user sessions:

• Scroll depth measures the percentage of the page scrolled
• Scroll speed measures the average scroll speed in pixels per second
• Time spent measures the total time spent on the page
• Content consumed measures the number of articles read, images viewed, etc.

We can use a regression model like linear regression or support vector regression to find a relationship between these variables. The model would learn to predict content consumed based on scroll depth, scroll speed, and time spent.

Consider a highly simplified regression model that finds the following relationship:

Content consumed = 0.5 * (Scroll depth) + 0.2 * (Scroll speed) + 0.3 * (Time spent) 

Now, if a new user scrolls 80% of the page with an average speed of 100 pixels per second and spends 60 seconds on the page, the model would predict:

Content consumed = 0.5 * (80) + 0.2 * (100) + 0.3 * (60) = 78 

This equation means the model predicts the user will consume approximately 78 units of content (e.g., read 78 articles, view 78 images, etc.).

Dwell Time Analysis

Here, we track how long a user spends on specific sections or elements within a page. This tracking helps us understand which content captures their attention and for how long.

Client Side Sample Implementation

Here, we first use JavaScript to track how long a user spends on each section of an article. We can use scroll events, visibility change events, or other techniques to monitor when a section enters and exits the viewport. Furthermore, we divide each article into logical sections (e.g., by headings, paragraphs, or other structural elements). These divisions allow us to analyze dwell time on a more granular level.

This example employs a scroll event listener:

// Get all the sections of the article 
const sections = document.querySelectorAll('article section'); 
// Initialize an array to store dwell time data for each section let dwellTimes = new Array(sections.length).fill(0); 
// Initialize an array to store the timestamps when each section becomes visible let sectionEntryTimestamps = new Array(sections.length).fill(null); 
// Function to track when a section enters and exits the 
// viewport.
const handleSectionVisibility = (entries) => { 
entries.forEach(entry => { 
const sectionIndex = Array.from(sections).indexOf(entry.target); 
if (entry.isIntersecting) { 
// Section has entered the viewport 
sectionEntryTimestamps[sectionIndex] = Date.now(); 
} else { 
// Section has exited the viewport 
if (sectionEntryTimestamps[sectionIndex] !== null) {
dwellTimes[sectionIndex] += Date.now() - sectionEntryTimestamps[sectionIndex]
sectionEntryTimestamps[sectionIndex] = null; 
} 
} 
}); 
}; 
// Create a new IntersectionObserver to observe section visibility
const observer = new IntersectionObserver(handleSectionVisibility, { threshold: 0.5, // Adjust the threshold as needed (e.g., 0.75 for 75% visibility) }); 
// Observe each section 
sections.forEach(section => { 
observer.observe(section); 
}); 
// Function to log the dwell time data (e.g., when the user leaves the page) const logDwellTimes = () => { 
console.log('Dwell Times:', dwellTimes); 
// You can send this data to your server for further analysis and processing }; 
// Add an event listener to trigger the logging function when the user leaves the page 
window.addEventListener('beforeunload', logDwellTimes);

The following example uses a visibilitychange event:

JavaScript
// Get all the sections of the article 
const sections = document.querySelectorAll('article section'); 
// Initialize an array to store dwell time data for each section 
let dwellTimes = new Array(sections.length).fill(0); 
// Initialize an array to store the timestamps when each section becomes visible
let sectionEntryTimestamps = new Array(sections.length).fill(null); 
// Function to track when a section's visibility changes 
const handleVisibilityChange = (event) => { 
const section = event.target; 
const sectionIndex = Array.from(sections).indexOf(section); 
if (document.visibilityState === 'visible') { 
// Section is now visible 
sectionEntryTimestamps[sectionIndex] = Date.now(); 
} else { 
// Section is now hidden (e.g., user switched tab, minimized window) 
if (sectionEntryTimestamps[sectionIndex] !== null) { 
dwellTimes[sectionIndex] += Date.now() - 
sectionEntryTimestamps[sectionIndex]; 
sectionEntryTimestamps[sectionIndex] = null; 
} 
} 
}; 
// Add a visibilitychange event listener to each section
sections.forEach(section => { 
section.addEventListener('visibilitychange', handleVisibilityChange); }); 
// Function to log the dwell time data (e.g., when the user leaves the page) 
const logDwellTimes = () => { 
console.log('Dwell Times:', dwellTimes); 
// You can send this data to your server for further analysis and processing }; 
// Add an event listener to trigger the logging function when the user leaves the page 
window.addEventListener('beforeunload', logDwellTimes); 

Server Side Sample Implementation

Once our server receives the dwell time data (such as total time spent per section, number of sections viewed, and overall page dwell time) sent from the client device (perhaps via navigator.sendBeacon or periodic pings), we can apply machine learning models. These models, such as regression or decision trees, analyze these dwell time patterns to predict user engagement and potential future content consumption.

Regression model example:

Let’s assume we are tracking the following data points derived from the client-side dwell time analysis for each user session on an article page:

• Average dwell time per viewed section (AvgDwell) is the average time in seconds spent actively viewing each section that entered the viewport.
• Total active page dwell time (TotalDwell) is the total time in seconds the user spent actively viewing any section on the page (sum of dwell times, excluding time when the tab was hidden).
• Number of sections viewed (SectionsViewed) is the count of distinct sections the user dwelled on for more than a minimal threshold (e.g., > 1 second).

We can use a regression model (like linear regression, or more complex models like gradient boosting regressors for better accuracy) to find a relationship between these dwell time features and the amount of content the user is likely interested in consuming next. This personalized pagination colab provides a walkthrough on how to build such a regression model with sample data.

Let’s say our trained regression model finds the following simplified linear relationship:

Predicted_Next_Content_Units = 5 + (0.8 * AvgDwell) + (0.1 * TotalDwell) + (2.0 * SectionsViewed)

Now, imagine a user visits a page and the server receives the following data:

• AvgDwell = 15 seconds
• TotalDwell = 70 seconds
• SectionsViewed = 4 sections

The model would predict the potential next content consumption:

Predicted_Next_Content_Units = 5 + (0.8 * 15) + (0.1 * 70) + (2.0 * 4) = 32

In other words, the model predicts the user will consume approximately 32 units of content (e.g., read 32 articles, view 32 images, etc.).

Conclusion

In today’s digital landscape, users expect seamless and engaging online experiences. Optimizing content delivery to users is not just a technical necessity but is also a key differentiator.

These requirements cannot be met with traditional pagination due to its rigid structure and potential for slow loading times. Therefore, by leveraging AI to intelligently analyze user behavior as we discussed in this article using scroll speed and depth tracking or dwell time analysis we can create a responsible (i.e., respecting users’ bandwidth and avoiding unnecessary data transfer) and personalized browsing experience for users.