Presearch

The Presearch vision paper laid out the architecture for how a decentralized search engine could work, and we’ve been building non-stop toward achieving that goal — from the core dev team, to open source contributors working on search packages, to the tens of thousands of community members sharing ideas, promoting the project, and running nodes powering the Presearch Testnet this past year.In order to launch the Presearch Mainnet, we needed to solve severalkey technical challenges related to node performance and reliability, in order to ensure that community-run nodes could provide an optimal search experience for end users while still promoting decentralization.<ol><li>End users expect fast search results, so making them wait on slow or unreliable nodes is not good.</li><li>End users expect a stable and reliable search experience, so fault tolerance needs to be built into the selection of nodes to avoid failures.</li><li>Nodes should be incentivized higher for good performance (low latency, high uptime, high success rate).</li><li>Search results should be as fast as possible within each market, meaning the nodes serving search results should be located geographically close to those running searches to minimize latency.</li></ol>The release of Smart Node Routing solves the first three of these requirements, with Geographically-distributed Gateways (to be covered in a subsequent post) solving the last one.Smart Node Routing is crucial for decreasing the latency of end-user searches, for properly incentivizing node operators to provide the most reliable nodes possible, and for ensuring the network maximizes the use of the best available nodes. In this article here, we’re going to break down what smart node routing is, how it works, and why it’s so critical for the project.<h3 id="tldr">TLDR;</h3>Prior to the introduction of Smart Node Routing, all nodes connected to a gateway server would be placed in a queue and receive the same number of requests so long as they remained connected. This means end users could arbitrarily get a fast/reliable node or a slow/unreliable node, and rewards were simply modified based upon each node’s reliability after the fact.With Smart Node Routing, now the number of searches each node runs is roughly proportional to the node’s reliability, and requests to nodes which are expected to fail may also be proactively dual-routed to backup nodes. This means end-user searches are much more likely to be handled by faster and more reliable nodes, and that even requests to less reliable nodes will rarely result in a noticeable performance degradation for the end user. This ability to optimize the network regardless of the presence of less reliable nodes within the network is a key requirement for running a high-performance decentralized search engine.<h3 id="what-is-smart-node-routing-and-how-does-it-work">What is Smart Node Routing and how does it work?</h3>Before we dive into Smart Node Routing, we must understand the role of two layers of the Presearch network that nodes connect to and rebalance between: gateways and gateway pools.<h2 id="header-0">What are node gateways and gateway pools?</h2>A gateway, also sometimes referred to a “gateway node” or “gateway server”, is a specialized node in the Presearch network that is responsible for anonymizing and routing end user requests from webservers to other nodes to fulfill searches. Every webserver (running the presearch.com website) attempts to connect to one gateway, and that gateway is one of many gateways within a gateway pool.Gateway pools are a collection of gateways, usually representing a geographic region. Presearch currently has five gateway pools: na-us-1, na-us-2, eu-de-1, and as-sg-1, and testnet, with plans to continue adding many more gateway pools over time across the world.<h2 id="header-1">How are traffic, webservers, gateways, and nodes balanced across gateway pools?</h2>Each gateway pool essentially acts as a subgraph or subnet within the Presearch network, allowing search requests from an end user to route to the closest webserver geographically, which is part of the closest gateway pool geographically, which has the best performing nodes for that gateway pool connected.In order to ensure balanced operation of the network, the number of end users searching in a particular region will ideally be balanced evenly across the webservers in that region’s gateway pool, the webservers will be balanced evenly across the gateways in that gateway pool, and the number of nodes will be balanced evenly across the gateways in that gateway pool. We’ll get more into how gateway pools operate and how traffic and nodes are routed and balanced between them in an upcoming article on Geographically-distributed Gateways.<h2 id="header-2">Where does Smart Node Routing occur?</h2>Smart node routing happens at the gateway level, which means that there isn’t a single global place where smart node routing occurs — each individual gateway receives search requests and routes them to the nodes connected to that gateway, with prioritization based upon the nodes’ reliability scores. The higher its reliability score, the more likely a node is to receive more search requests as long as it remains connected.So for example, if there are 3 nodes that have a reliability score of 100, 50 &amp; 1, the node that has a reliability score of 50 will currently receive approximately 50x the searches over time versus the node that has a reliability score of 1. The node that has a reliability score of 100 will receive approximately 2x the searches of the node with reliability of 50 over time. This means that instead of the gateways routing an equal amount of searches to each node, that higher-performing nodes now receive more searches based on reliability. Note that these relative differences in number of searches between different reliability scores may adjust over time, but the above examples represent the ratios as of Mainnet launch.<h2 id="header-3">Will rewards now change to nodes based upon their reliability scores?</h2>While Smart Node Routing just launched in Mainnet, it has actually already been enabled on the Presearch Testnet since early April, before we started production traffic tests on the Testnet (“soft launch”) in preparation for Mainnet launch last week. While the rewards calculation and rewards per search definitely change, the net rewards AMOUNTS each node receives should be virtually no different.This is because prior to Smart Node Routing, every node connected to a gateway received approximately the same traffic, but then had the rewards per search rate multiplied by the reliability score. For example, a node with a reliability score of 50 would receive half the reward of a node with a reliability score of 100, and a node with a reliability of 1 would receive one percent of the reward of a node with a reliability score of 100. This can be seen in the original “utilization” part of the node reward calculation <a href="https://news.presearch.io/presearch-node-rewards-are-here-a425508df095" rel="noopener noreferrer nofollow">found here</a>:<img src="https://academy-public.coinmarketcap.com/srd-optimized-uploads/3cc5dc8f94074faebf0794b9d7a32acb.png"> As of the release of Smart Node Routing, the node_quality_score (“reliability score” as a percentage) is now factored into the per_node_successful_requests on each gateway, since the number of requests actually received is determined by the reliability score, so the the calculation has now barely been changed from:utilization_% = ( average_requests_per_node / all_nodes_total_requests) * (node_quality_score)to now be:per_node_successful_requests = (node_quality_score * average_requests_per_node) / all_nodes_total_requestswhere the bold portion takes place at query time, and the rest of the calculation takes place later when calculating rewards for the reward period. While previously per_node_successful_requests was approximately the average number of requests per node, it has now been modified so that the reliability score is used as a multiplier to determine the number of requests routed to a node by Smart Node Routing.As you can see though, the results of the calculation are exactly the same mathematically before and after the change. Given this, while we moved WHERE in the network the reliability score (node_quality_score) is factored in, the effective reward calculation remains unchanged. It has just been pushed upstream proactively to the routing of searches to nodes.If you’d like a refresher on the full rewards calculation, you can find it in section 7.3 of the <a href="https://presearch.io/vision.pdf" rel="noopener noreferrer nofollow">Presearch Vision Paper</a>, though we’ll have an updated version posted soon which contains some additional normalizations now present for handling rewards spread across geographically-distributed gateway pools.<h2 id="header-4">Rewardable vs. Successful vs. Attempted requests</h2>One of the most crucial focuses for any compelling web search engine on is latency, and this of even greater concern for a decentralized search engine. If requests takes too long, users will take their searches elsewhere. That’s why in Smart Node Routing, there is a separate calculation keeping track of each node’s latency, and for low-reliability nodes, the network can now proactively use high-reliability/low-latency nodes as backup nodes. This allows a gateway, when it is routing a request to a node with a high likelihood of failure, to proactively dual-route the query to a better-performing node to ensure a timely response is received and the user experience is still good, even if the primary node fails. The primary node will still get credit for the search if it completes it in time, and the backup node’s total payouts won’t be affected in this case, since it will still receive its normal search traffic in addition to any assistance it may provide as a backup node.In order to achieve this optimal balance between “best end user experience” and “all nodes get a fair shot to perform searches on the network”, we introduced a few distinctions in the stats between searches:<img src="https://academy-public.coinmarketcap.com/srd-optimized-uploads/24068459e15e46f79d09f169e70a509c.png">The “Attempted Requests” shows the total number of search requests received and the “Successful Requests” shows how many of those were successful. The “Rewardable” requests, however, shows the total number of requests that were eligible for a reward based upon being either primary requests (your node’s turn to run a search) or backup requests that succeeded when a different primary node fails (your node got credit instead). If your node served as a backup node and the primary node succeeded, it will NOT be recorded as a rewardable request. To prevent confusion, please note that “rewardable” does not mean “rewarded”. It just means that IF your node is eligible to receive rewards, that the request received a reward. For example, if you run a node without a PRE stake, and it receives a request, that is a “rewardable” request, but unfortunately no reward will actually be paid out because the node doesn’t have a stake.“Rewardable Requests” are reported separately from “Attempted Requests” to provide full insight to you, as a node operator, about how your node is performing and so you can related each request to the detailed performance statistics reported each period, including the total rewards paid.<h3 id="gateway-pool-rebalancing">Gateway pool rebalancing</h3>While this post is focused on search traffic routing within a gateway, routing across gateways and gateway pools also plays a critical role in the Presearch network. As mentioned earlier, automatic rebalancing of nodes and webservers occurs on an ongoing basis between different gateways within a gateway pool, and rebalancing of nodes across gateway pools to handle end user traffic fluctuations will also play a critical role going forward. We’ll have another upcoming post describing how these mechanisms work in the coming weeks.For now, hopefully this post provided some deeper technical insight into how Smart Node Routing helps the Presearch network intelligently balance between the need for a diverse and globally-decentralized network of nodes and the need to optimize for both end-user search latency and the fair distribution of searches and rewards for each node supporting the network.If you have questions, please join one of our Community links belowRun a node: <a href="https://nodes.presearch.org/" rel="noopener noreferrer nofollow">https://nodes.presearch.org</a> Keyword Stake: <a href="https://keywords.presearch.org/" rel="noopener noreferrer nofollow">https://keywords.presearch.org</a> Info site: <a href="https://presearch.io/" rel="noopener noreferrer nofollow">https://presearch.io</a> Search with Presearch: <a href="https://presearch.com/" rel="noopener noreferrer nofollow">https://presearch.com</a>Community channels:Telegram: t.me/Presearch Twitter: <a href="https://twitter.com/presearchnews" rel="noopener noreferrer nofollow">https://twitter.com/presearchnews</a> YouTube: <a href="https://www.youtube.com/c/Presearch/videos" rel="noopener noreferrer nofollow">https://www.youtube.com/c/Presearch/videos</a> Discord: <a href="https://discord.gg/8TCX8WJE" rel="noopener noreferrer nofollow">https://discord.gg/8TCX8WJE</a> Minds: <a href="https://www.minds.com/presearch/" rel="noopener noreferrer nofollow">https://www.minds.com/presearch</a> Reddit: <a href="https://www.reddit.com/r/Presearch/" rel="noopener noreferrer nofollow">https://www.reddit.com/r/Presearch/</a> Flote: <a href="https://flote.app/user/presearch" rel="noopener noreferrer nofollow">https://flote.app/user/presearch</a> Twitter (Community): <a href="https://twitter.com/TeamPresearch" rel="noopener noreferrer nofollow">https://twitter.com/TeamPresearch</a> Telegram(News): t.me/PresearchNews Facebook: <a href="https://www.facebook.com/presearch.io" rel="noopener noreferrer nofollow">https://www.facebook.com/presearch.io</a> Email: support@presearch.ioInterested in partnering? Send us an email at — promote@presearch.io

Smart Node Routing

Table of Contents

Presearch

Other articles published on Jun 8, 2022

Part 3: NFTs — A Non-Fungible Renaissance

A HOW-TO-BE-RICH GUIDE IN FISH UNCHAINED

Yellow Network’s $YELLOW Token Sale Comes to Launchblock.com

LBP Farm — everything you need to know

STEPN’s Action Plan for Game Economy

Connectivity in Africa: what’s behind the issue and what to consider to upgrade the scenario

Join our free newsletter for daily crypto updates!