Presearch

This post is targeted toward node operators and focuses on how to optimize your nodes for best results as we roll out geographically-distributed node gateways and decentralize the node network.As Mainnet approaches, we encourage node operators to continue decentralizing your Presearch nodes to meet the needs of searchers as the network footprint becomes more distributed.To assist with this transition, we are providing some additional information on the current utilization of the network — where search requests originate from and the geographic location of current Testnet nodes in particular.As the network expands to support regional gateway pools, nodes should not all be in one geographical area, using the same hosting provider, using the same IP ranges, or all having the same server specifications. Below we will discuss the necessity of avoiding these behaviors to better decentralize the node network.But first, some stats on current supply and demand for Presearch processing resources.<h2 id="header-0">Presearch node distribution globally</h2><h2 id="header-1"></h2><img src="https://academy-public.coinmarketcap.com/srd-optimized-uploads/c8713918059c4281b49992cd704899c8.png"><h2 id="header-2">Compared to Presearch search distribution globally</h2><h2 id="header-3"></h2><img src="https://academy-public.coinmarketcap.com/srd-optimized-uploads/130668671e6a4953b7e4c0cbe5a536bc.png">While the US accounts for a significant portion of search traffic, there are significantly more global searchers than there are node operators.<h2 id="header-4">The importance of geographical diversification</h2>Lack of geographical diversity negatively impacts latency of search results and usefulness of those nodes to handle traffic in other regional gateways if too many nodes are all centralized in one region.Some node operators have a single node, and others have hundreds. Many of those with hundreds have spread them out very well across service providers (good 🕺) and different regions (good🕺) to help promote decentralization and to ensure that their nodes aren’t all subjected to problems if a service provider blocks them or the service provider themselves gets blocked.Others have signed up for dozens or hundreds of nodes all in one geographical location (not ideal) and all with one hosting provider (not ideal). The former will hurt the latency score (and ultimately reliability score) of nodes as new gateways come online, and the latter could lead to nasty surprises if the hosting provider or a federated resource blocks the nodes or traffic by IP address.If you’re looking to further decentralize your node footprint, here’s a map of current node locations (blue dots) across the globe. You might try filling in some of the areas currently lacking nodes.<img src="https://miro.medium.com/max/1400/1*dMckxTDFG025nfk-rHIYYg.png">The node gateways at Mainnet launch have been announced as:na-us-1: North America — California, USAna-us-2: North America — Ohio, USAeu-de-1: Europe — Frankfurt, Germanyas-sg-1: Asia — SingaporeAdditional gateways will roll out over time to both speed up end-user searches and incentivize further geographical decentralization of nodes.<h2 id="header-5">The importance of hardware diversification</h2>While this isn’t a major concern at the moment, as we expand the range of roles for nodes, and as traffic grows, it will be something that you’ll need to take into account.We bring it up now so that those who may be making plans to migrate existing nodes or to set up new ones during the transition to Mainnet will be able to take these factors into consideration and future-proof their nodes for the network changes that are sure to come.Because we plan to bundle the software to fulfill multiple node roles within the same Docker image, as new roles are available, you will be able to activate additional functionality through the node dashboard or through issuing a command to your node. As such, keep in mind that different hardware profiles may be beneficial for different types of roles.Nodes will ultimately be designed to perform many functions (federating from other search engines and APIs, indexing, searching indexes, crawling, coordinating and relaying requests to other nodes, data storage, consensus, training AI models, etc.).If too many of the nodes all share the same hardware / network profile (amount of disk space, amount of RAM, CPU speed, data center location), they will be more likely to “fail” in the same ways at the same time, making the network more fragile. Some examples of potential scenarios that could cause issues:— Nodes are handling a lot of network traffic and hit some arbitrary “network bandwidth limit” set by a bargain VPS plan and the provider then terminates the server.— A search engine or API that is being federated blocks an IP range for a service provider that is originating too much traffic.— A lack of disk space means that nodes can’t store crawl data, indexes, etc. once those roles are available.Additionally, if all of the nodes are too similar, the network won’t have a diversity of performance profiles to optimize against as those roles become available.Again, this is mainly forward-looking, and we do not recommend that you go out and spend a significant amount on a high capacity node, anticipating future usage that doesn’t come for some time. We mainly just want you to keep this in mind in case you have the opportunity to diversify your nodes at little or no additional cost.Testnet vs. MainnetIn Testnet, the primary focus has been on developing nodes, increasing stability of the network, handling many node connections, building out rewards and reporting functionality and optimizing traffic distribution. After Mainnet, the focus will primarily be on ensuring the network delivers the best end user experience (fastest, most stable, least errors).As such:A. By default, nodes will be assigned to the gateway pool (geographical region) in order of:1) Where their latency outperforms enough of the other nodes needed on the gateway pool2) Where the node will get the best performance (latency + success rate) after (1) is satisfiedB. This means that if too high a ratio of nodes is located in a particular location/service provider, that those nodes will all be competing with each other for best gateway pool placement, likely resulting in the nodes not getting their optimal placement (and thus getting lower latency scores and less traffic and rewards) because they are not well decentralized.<h2 id="header-6">If a node has issues during the Mainnet transition</h2>Should a node have an issue that makes it unsuitable for use on Mainnet — likely due to reliability issues, or it being blocked by a federated resource through an IP blacklist or rate limiting of an IP range— the node will automatically be transitioned from Mainnet back to Testnet.Nodes currently access multiple data sources in the federating role — existing search engines, APIs and databases — and if they are not able to return results from ALL sources, then they will not be considered successful requests on Mainnet. Unsuccessful requests can result in the node being migrated back to Testnet.While on Testnet, nodes will continue to accrue normalized rewards comparable to what they would receive on Mainnet if successful there, ensuring that there is little to no economic penalty for running on Testnet. This may change over time, but during the transition period and until after Mainnet live for some time, this will be the case.<h2 id="header-7">The “early access” role of Testnet going forward</h2>If your node gets migrated back to Testnet, take heart in knowing that Testnet nodes are still required and critical for testing the network and that Testnet nodes will get the benefit of new updates and features before Mainnet. Running a node on Testnet means your node will also be first in line for updates and new node roles as they roll out. While Testnet nodes will receive less traffic, their rewards per search will be higher to keep them inline with net reward payouts for Mainnet nodes and ensure Testnet nodes aren’t penalized for serving in this ‘early access’ and more experimental state. Some node operators may even prefer to run primarily Testnet nodes for these reasons.Nodes encountering errors, timeouts, or other issues being moved to Testnet will also enable focused development on improvements for those nodes on the Testnet without disrupting the end user experience for those running searches.Additionally, we do plan — over time — to enable Mainnet nodes to return partial results (like Testnet nodes can currently) and to receive pro-rata reward compensation for incomplete results, we just did not have time to implement that functionality before the scheduled Mainnet launch. So, in the near future, if your node is able to return results from most federated sources, but not all, then it would have the option to run on the Testnet despite partial failures.For those of you who have prepaid into long-term contracts for VPS service, please know that your existing nodes that are not suitable for the federating role on Mainnet may instead be used for additional roles (crawling, indexing, etc.) as those roles become available, since most blocks or other issues that currently result in incomplete or timed-out results will likely not affect the fulfillment of other those other node roles.<h3 id="tldr">TLDR;</h3>If you’re running multiple nodes, please focus on decentralizing both your geographical region and service providers to avoid having all of your nodes subject to similar performance and/or failure scenarios. While all nodes will be part of the Mainnet launch, some nodes may get kicked back to the Testnet after launching on Mainnet if they are struggling to deliver full results or timing out, or may get placed in gateway pools that reduce their latency if they are too heavily condensed in the same geographical areas as other node operators. Decentralizing your nodes is highly recommended to both prevent sub-optimal performance of your own nodes and to promote the overall robustness of the Presearch network.Diversity of nodes is incredibly important for a decentralized search engine, and we appreciate node operators doing everything they can to assist in decentralizing the network to provide searchers with a great user experience.Thanks to everyone for your support as we undertake this important transition from Testnet to Mainnet!<h3 id="if-you-have-questions-please-join-one-of-our-community-links-below">If you have questions, please join one of our Community links below</h3>Run a node: <a href="https://nodes.presearch.org/" rel="noopener noreferrer nofollow">https://nodes.presearch.org</a>Keyword Stake: <a href="https://keywords.presearch.org/" rel="noopener noreferrer nofollow">https://keywords.presearch.org</a>Info site: <a href="https://presearch.io/" rel="noopener noreferrer nofollow">https://presearch.io</a>Search with Presearch: <a href="https://presearch.com/" rel="noopener noreferrer nofollow">https://presearch.com</a>Community channels:Telegram: t.me/PresearchTwitter: <a href="https://twitter.com/presearchnews" rel="noopener noreferrer nofollow">https://twitter.com/presearchnews</a>YouTube: <a href="https://www.youtube.com/c/Presearch/videos" rel="noopener noreferrer nofollow">https://www.youtube.com/c/Presearch/videos</a>Discord: <a href="https://discord.presearch.com/" rel="noopener noreferrer nofollow">https://discord.presearch.com</a>Minds: <a href="https://www.minds.com/presearch/" rel="noopener noreferrer nofollow">https://www.minds.com/presearch</a>Reddit: <a href="https://www.reddit.com/r/Presearch/" rel="noopener noreferrer nofollow">https://www.reddit.com/r/Presearch/</a>Flote: <a href="https://flote.app/user/presearch" rel="noopener noreferrer nofollow">https://flote.app/user/presearch</a>Twitter (Community): <a href="https://twitter.com/TeamPresearch" rel="noopener noreferrer nofollow">https://twitter.com/TeamPresearch</a>Telegram(News): t.me/PresearchNewsFacebook: <a href="https://www.facebook.com/presearch.io" rel="noopener noreferrer nofollow">https://www.facebook.com/presearch.io</a>Email: support@presearch.ioInterested in partnering? Send us an email at — promote@presearch.io

The transition to Mainnet — decentralization & optimization

Table of Contents

Presearch

Other articles published on May 21, 2022

Upcoming Priority Projects on Kommunitas

Pyramid Airdrop Event

Kommunitas x Blockster Campaign Giveaway Rewards Distributed

Our INO with iStep has Sold Out!

Tutorial on How to Participate in Kommunitas IKOs on The Solana Network

200K Followers Twitter Event

Join our free newsletter for daily crypto updates!