{"id":13741,"date":"2025-12-16T12:52:19","date_gmt":"2025-12-16T07:22:19","guid":{"rendered":"https:\/\/www.youstable.com\/blog\/?p=13741"},"modified":"2025-12-24T16:14:02","modified_gmt":"2025-12-24T10:44:02","slug":"optimize-elasticsearch-on-linux","status":"publish","type":"post","link":"https:\/\/www.youstable.com\/blog\/optimize-elasticsearch-on-linux","title":{"rendered":"How to Optimize ElasticSearch on Linux Server for Better Speed"},"content":{"rendered":"\n<p>To optimize Elasticsearch on Linux server, allocate the JVM heap to ~50% of RAM (max 31 GB), use SSD\/NVMe storage, disable swap and Transparent Huge Pages, tune vm.max_map_count and ulimits, design shards correctly (20\u201350 GB each), and adjust index settings (refresh interval, replicas) for your workload while monitoring GC, I\/O, and caches.<\/p>\n\n\n\n<p>Optimizing Elasticsearch on a Linux server means tuning the OS, JVM, and Elasticsearch settings together. In this guide, I\u2019ll walk you through practical, production-proven steps to optimize performance, stability, and cost. <strong>Whether you\u2019re indexing logs or powering search, these best practices help you avoid bottlenecks and scale smoothly.<\/strong><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"quick-checklist-elasticsearch-optimization-on-linux\"><strong>Quick Checklist: Elasticsearch Optimization on Linux<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use 64-bit Linux, SSD\/NVMe, and adequate RAM\/CPU.<\/li>\n\n\n\n<li>Set JVM heap to ~50% of RAM (up to 31 GB). Leave the rest for the OS <a href=\"https:\/\/www.youstable.com\/blog\/clear-cache-in-windows-11\/\">file cache<\/a>.<\/li>\n\n\n\n<li>Disable swap and Transparent Huge Pages (THP). Lock memory.<\/li>\n\n\n\n<li>Increase vm.max_map_count and file descriptors (ulimits).<\/li>\n\n\n\n<li>Right-size shards (20\u201350 GB each) and replicas based on traffic and SLA.<\/li>\n\n\n\n<li>Use keyword fields for aggregations; avoid fielddata on text.<\/li>\n\n\n\n<li><strong>For bulk indexing:<\/strong> replicas=0, larger refresh_interval, and _bulk API batches 5\u201315 MB.<\/li>\n\n\n\n<li>Monitor GC, I\/O wait, cache hit rates, and slow logs. Scale nodes or rebalance shards.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"prepare-the-linux-server\"><strong>Prepare the Linux Server<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"hardware-and-storage\"><strong>Hardware and Storage<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Storage: <\/strong>Prefer NVMe or SSD. Elasticsearch is I\/O-intensive.<\/li>\n\n\n\n<li><strong>CPU:<\/strong> Fewer fast cores often beat many slow ones for search latency.<\/li>\n\n\n\n<li><strong>RAM: <\/strong>Size for your data and query patterns. Heap max 31 GB; OS page cache needs room to breathe.<\/li>\n\n\n\n<li><strong>Filesystem: <\/strong>XFS or ext4. Consider noatime to reduce write amplification.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"kernel-and-sysctl-tuning\"><strong>Kernel and sysctl Tuning<\/strong><\/h3>\n\n\n\n<p>Elasticsearch maps many files into memory. Increase map count, set conservative swappiness, and allow ample file handles.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/sysctl.d\/99-elasticsearch.conf\nvm.max_map_count=262144\nvm.swappiness=1\nfs.file-max=2097152\nnet.core.somaxconn=65535\n\n# Apply immediately\nsysctl --system<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"disable-swap-and-transparent-huge-pages-thp\"><strong>Disable Swap and Transparent Huge Pages (THP)<\/strong><\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code># Disable swap now\nswapoff -a\n# Comment out swap in \/etc\/fstab to persist\nsed -ri 's\/^\\s*(&#91;^#].*\\s+swap\\s+)\/#\\1\/' \/etc\/fstab\n\n# Disable THP via systemd unit\ncat &gt;\/etc\/systemd\/system\/disable-thp.service &lt;&lt;'EOF'\n&#91;Unit]\nDescription=Disable Transparent Huge Pages\nAfter=sysinit.target local-fs.target\n\n&#91;Service]\nType=oneshot\nExecStart=\/bin\/sh -c 'echo never &gt; \/sys\/kernel\/mm\/transparent_hugepage\/enabled'\nExecStart=\/bin\/sh -c 'echo never &gt; \/sys\/kernel\/mm\/transparent_hugepage\/defrag'\nRemainAfterExit=yes\n\n&#91;Install]\nWantedBy=multi-user.target\nEOF\n\nsystemctl daemon-reload\nsystemctl enable --now disable-thp<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"file-descriptors-and-memory-locking\"><strong>File Descriptors and Memory Locking<\/strong><\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/security\/limits.d\/elasticsearch.conf\nelasticsearch soft nofile 65536\nelasticsearch hard nofile 65536\nelasticsearch soft nproc  65536\nelasticsearch hard nproc  65536\nelasticsearch soft memlock unlimited\nelasticsearch hard memlock unlimited\n\n# Systemd override for the service\nsystemctl edit elasticsearch\n# Add:\n&#91;Service]\nLimitNOFILE=65536\nLimitNPROC=65536\nLimitMEMLOCK=infinity<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"filesystem-mount-options\"><strong>Filesystem Mount Options<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use noatime on data volumes to reduce metadata writes.<\/li>\n\n\n\n<li>On SSD\/NVMe, keep readahead conservative (e.g., 128): <code>blockdev --setra 128 \/dev\/nvme0n1<\/code>.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"tune-the-jvm-and-elasticsearch-process\"><strong>Tune the JVM and Elasticsearch Process<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"set-the-heap-size-correctly\"><strong>Set the Heap Size Correctly<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>General rule: <\/strong>Xms = Xmx = ~50% of system RAM up to 31 GB (to preserve compressed OOPs and better GC).<\/li>\n\n\n\n<li>Leave the other 50% for the OS page cache, file system, and Lucene.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/elasticsearch\/jvm.options.d\/heap.options\n-Xms16g\n-Xmx16g<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"garbage-collector-and-jvm-flags\"><strong>Garbage Collector and JVM Flags<\/strong><\/h3>\n\n\n\n<p>Elasticsearch 8.x uses G1GC by default. These flags are safe, production-friendly defaults.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/elasticsearch\/jvm.options.d\/gc.options\n-XX:+UseG1GC\n-XX:G1ReservePercent=15\n-XX:InitiatingHeapOccupancyPercent=30\n-XX:+AlwaysPreTouch\n-XX:+HeapDumpOnOutOfMemoryError\n-XX:HeapDumpPath=\/var\/lib\/elasticsearch\/heapdump.hprof\n-Xlog:gc*:file=\/var\/log\/elasticsearch\/gc.log:time,uptime:filecount=5,filesize=64m<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"lock-memory\"><strong>Lock Memory<\/strong><\/h3>\n\n\n\n<p>Prevent the OS from swapping out Elasticsearch memory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/elasticsearch\/elasticsearch.yml\nbootstrap.memory_lock: true\n\n# Ensure systemd allows it (see LimitMEMLOCK above), then:\nsystemctl daemon-reload\nsystemctl restart elasticsearch<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"production-elasticsearch-yml-essentials\"><strong>Production elasticsearch.yml Essentials<\/strong><\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/elasticsearch\/elasticsearch.yml\ncluster.name: prod-es\nnode.name: node-1\nnode.roles: &#91; data, ingest ]   # Separate master-eligible nodes for stability\npath.data: &#91; \/data1\/es, \/data2\/es ]\npath.logs: \/var\/log\/elasticsearch\n\nnetwork.host: 0.0.0.0\nhttp.port: 9200\n\ndiscovery.seed_hosts: &#91; \"10.0.0.11\", \"10.0.0.12\", \"10.0.0.13\" ]\ncluster.initial_master_nodes: &#91; \"node-1\", \"node-2\", \"node-3\" ]\n\n# Safe, helpful defaults\nindices.memory.index_buffer_size: 20%\ncluster.routing.allocation.disk.watermark.low: 85%\ncluster.routing.allocation.disk.watermark.high: 90%\n\n# Advanced; change only if you know the impact\n# indices.breaker.total.limit: 70%\n# thread_pool.write.queue_size: 1000<\/code><\/pre>\n\n\n\n<p>Secure clusters with TLS and authentication (x-pack security). While it\u2019s mostly a security feature, preventing unauthorized heavy queries also protects performance.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"design-shards-and-indices-for-speed\"><strong>Design Shards and Indices for Speed<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"right-size-primaries-and-replicas\"><strong>Right-Size Primaries and Replicas<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Shard size target: <\/strong>20\u201350 GB per shard for general use. Too many small shards wastes heap and file handles.<\/li>\n\n\n\n<li><strong>Primaries: <\/strong>Match to data volume and node count; avoid creating dozens of primaries without need.<\/li>\n\n\n\n<li><strong>Replicas: <\/strong>1 replica is a good starting point for HA; set 0 during bulk loads and restore after.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"mappings-that-avoid-fielddata-traps\"><strong>Mappings That Avoid Fielddata Traps<\/strong><\/h3>\n\n\n\n<p>Use keyword for aggregations and exact matches; use text for full-text search. Don\u2019t aggregate on text fields (it triggers heavy fielddata).<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>PUT _index_template\/logs_template\n{\n  \"index_patterns\": &#91;\"logs-*\"],\n  \"template\": {\n    \"settings\": {\n      \"number_of_shards\": 3,\n      \"number_of_replicas\": 1,\n      \"refresh_interval\": \"1s\"\n    },\n    \"mappings\": {\n      \"properties\": {\n        \"message\": { \"type\": \"text\" },\n        \"service\": { \"type\": \"keyword\", \"doc_values\": true },\n        \"status\":  { \"type\": \"keyword\" },\n        \"bytes\":   { \"type\": \"long\" },\n        \"ts\":      { \"type\": \"date\" }\n      }\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"index-sorting-refresh-and-segment-management\"><strong>Index Sorting, Refresh, and Segment Management<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Index sorting (on timestamp or frequently filtered fields) speeds range queries.<\/li>\n\n\n\n<li>During heavy indexing, use a higher refresh_interval (e.g., 30s) or -1 temporarily.<\/li>\n\n\n\n<li>Force-merge to a small number of segments only for read-only indices (e.g., after bulk loads). It is resource-intensive.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>PUT logs-2025.01\/_settings\n{\n  \"index.routing.allocation.require._tier_preference\": \"data_hot\",\n  \"index.refresh_interval\": \"30s\",\n  \"index.sort.field\": &#91;\"ts\"],\n  \"index.sort.order\": &#91;\"asc\"]\n}<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"use-ilm-and-rollover\"><strong>Use ILM and Rollover<\/strong><\/h3>\n\n\n\n<p>Index Lifecycle Management (ILM) keeps shard sizes healthy and moves data from hot to warm\/cold nodes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>PUT _ilm\/policy\/logs_policy\n{\n  \"policy\": {\n    \"phases\": {\n      \"hot\": { \"actions\": { \"rollover\": { \"max_age\": \"1d\", \"max_size\": \"50gb\" } } },\n      \"warm\": { \"actions\": { \"allocate\": { \"include\": { \"_tier_preference\": \"data_warm\" } } } },\n      \"cold\": { \"actions\": { \"allocate\": { \"include\": { \"_tier_preference\": \"data_cold\" } } } }\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"speed-up-bulk-indexing\"><strong>Speed Up Bulk Indexing<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"refresh-interval-replicas-and-bulk-size\"><strong>Refresh Interval, Replicas, and Bulk Size<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Temporarily set replicas to 0 and refresh_interval to 30s or -1 for large ingests.<\/li>\n\n\n\n<li>Use the _bulk API with batches around 5\u201315 MB (not number of documents); parallelize with multiple workers.<\/li>\n\n\n\n<li>Prefer auto-generated IDs to avoid costly updates on the same document.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code># Before bulk\nPUT mydata\/_settings\n{ \"index\": { \"number_of_replicas\": 0, \"refresh_interval\": \"30s\" } }\n\n# Bulk ingest with proper batching (pseudo command)\ncurl -s -H \"Content-Type: application\/x-ndjson\" -XPOST localhost:9200\/_bulk --data-binary @batch.ndjson\n\n# After bulk\nPUT mydata\/_settings\n{ \"index\": { \"number_of_replicas\": 1, \"refresh_interval\": \"1s\" } }<\/code><\/pre>\n\n\n\n<p>If durability can be relaxed temporarily, setting translog durability to async during bulk can help, but it risks data loss on crashes. Use with caution.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"accelerate-search-queries\"><strong>Accelerate Search Queries<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"write-efficient-queries\"><strong>Write Efficient Queries<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Prefer filters (term, range) that can <a href=\"https:\/\/www.youstable.com\/blog\/fix-leverage-browser-caching\/\">leverage caches<\/a>; filter context doesn\u2019t affect scoring.<\/li>\n\n\n\n<li>Avoid leading wildcards on keyword fields. Use n-grams or prefix fields for autocomplete.<\/li>\n\n\n\n<li>Use search_after for deep pagination; from\/size gets expensive at high offsets.<\/li>\n\n\n\n<li>Pre-filter shards with index sorting and time-based indices to avoid hitting all shards.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"aggregation-and-field-choices\"><strong>Aggregation and Field Choices<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Aggregate on keyword\/numeric\/date fields with doc_values (default). Avoid aggregations on text.<\/li>\n\n\n\n<li>Disable norms on keyword fields to save memory.<\/li>\n\n\n\n<li>Minimize the number of concurrent heavy aggregations; consider composite aggregations or downsampling.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"monitor-benchmark-and-troubleshoot\"><strong>Monitor, Benchmark, and Troubleshoot<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"key-metrics-to-watch\"><strong>Key Metrics to Watch<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Heap usage and GC pauses (gc.log, _nodes\/stats). Sustained heap near 75\u201385% signals pressure.<\/li>\n\n\n\n<li>I\/O wait and disk throughput (iostat, sar). High I\/O wait = storage bottleneck.<\/li>\n\n\n\n<li>Query and indexing throughput\/latency (_cat\/thread_pool, _cat\/nodes, slow logs).<\/li>\n\n\n\n<li>Shard counts per node and average shard size (_cat\/shards). Too many shards hurts stability.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code># Useful commands\ncurl -s localhost:9200\/_cat\/health?v\ncurl -s localhost:9200\/_cat\/nodes?v\ncurl -s localhost:9200\/_cat\/shards?v\ncurl -s localhost:9200\/_nodes\/stats\/jvm,fs,indices?pretty<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"slow-logs-and-hot-warm-cold-architecture\"><strong>Slow Logs and Hot\u2013Warm\u2013Cold Architecture<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enable slow logs for index and search to pinpoint expensive queries or mappings.<\/li>\n\n\n\n<li>Use hot nodes for writes, warm for queryable historical data, cold for infrequent access; move indices with ILM.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"common-pitfalls-to-avoid\"><strong>Common Pitfalls to Avoid<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Oversharding:<\/strong> Thousands of tiny shards waste heap and file handles.<\/li>\n\n\n\n<li><strong>Too-large heap:<\/strong> Going above ~31 GB often worsens GC and reduces performance.<\/li>\n\n\n\n<li><strong>Fielddata on text: <\/strong>Explodes memory usage; use keyword for aggregations.<\/li>\n\n\n\n<li><strong>Unbounded wildcards and deep pagination: <\/strong>Rewrite queries for efficiency.<\/li>\n\n\n\n<li><strong>Leaving swap\/THP enabled: <\/strong>Causes jitter and instability under load.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"when-to-scale-vs-tune\"><strong>When to Scale vs. Tune<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>If heap is consistently high but GC is healthy, consider more data nodes or fewer shards per node.<\/li>\n\n\n\n<li>If I\/O wait dominates, upgrade to NVMe or distribute data across more disks\/nodes.<\/li>\n\n\n\n<li>If query latency spikes on specific aggregations, revisit mappings and pre-aggregation strategies.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"managed-option-let-youstable-handle-it\"><strong>Managed Option: Let YouStable Handle It<\/strong><\/h2>\n\n\n\n<p>Don\u2019t want to babysit JVM flags, sysctl, and shard balancing? At YouStable, our <a href=\"https:\/\/www.youstable.com\/blog\/benefits-of-fully-managed-dedicated-server\/\">Managed VPS and Dedicated Servers<\/a> run on enterprise SSD\/NVMe with tuned kernels, production file systems, and 24\u00d77 engineers who configure Elasticsearch for your workload\u2014heap sizing, ILM, hot\u2013warm\u2013cold tiers, monitoring, and ongoing optimization included.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"practical-configuration-examples\"><strong>Practical Configuration Examples<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"create-a-production-ready-index\"><strong>Create a Production-Ready Index<\/strong><\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>PUT myindex\n{\n  \"settings\": {\n    \"number_of_shards\": 3,\n    \"number_of_replicas\": 1,\n    \"refresh_interval\": \"1s\"\n  },\n  \"mappings\": {\n    \"properties\": {\n      \"title\":   { \"type\": \"text\" },\n      \"title.raw\": { \"type\": \"keyword\", \"ignore_above\": 256 },\n      \"category\": { \"type\": \"keyword\" },\n      \"price\":    { \"type\": \"scaled_float\", \"scaling_factor\": 100 },\n      \"created_at\": { \"type\": \"date\" }\n    }\n  }\n}<\/code><\/pre>\n\n\n\n<p><strong>Enable Slow Logs<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>PUT myindex\/_settings\n{\n  \"index.search.slowlog.threshold.query.warn\": \"5s\",\n  \"index.search.slowlog.threshold.fetch.warn\": \"1s\",\n  \"index.indexing.slowlog.threshold.index.warn\": \"1s\"\n}<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"faqs-optimize-elasticsearch-on-linux\"><strong>FAQs: Optimize ElasticSearch on Linux <\/strong><\/h2>\n\n\n\t\t<section\t\thelp class=\"sc_fs_faq sc_card    \"\n\t\t\t\t>\n\t\t\t\t<h3 id=\"what-is-the-best-jvm-heap-size-for-elasticsearch\">What is the best JVM heap size for Elasticsearch?<\/h3>\t\t\t\t<div>\n\t\t\t\t\t\t<div class=\"sc_fs_faq__content\">\n\t\t\t\t\n\n<p>Set Xms and Xmx to the same value, about 50% of system RAM, capped at 31 GB. This preserves compressed object pointers and keeps GC efficient. Leave the remaining RAM for the OS cache and Lucene.<\/p>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section\t\thelp class=\"sc_fs_faq sc_card    \"\n\t\t\t\t>\n\t\t\t\t<h3 id=\"how-many-shards-should-my-index-have\">How many shards should my index have?<\/h3>\t\t\t\t<div>\n\t\t\t\t\t\t<div class=\"sc_fs_faq__content\">\n\t\t\t\t\n\n<p>Aim for shards in the 20\u201350 GB range. Use enough primaries to distribute load across data nodes but avoid creating too many small shards. ILM with rollover by size\/time helps maintain healthy shard sizes automatically.<\/p>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section\t\thelp class=\"sc_fs_faq sc_card    \"\n\t\t\t\t>\n\t\t\t\t<h3 id=\"should-i-disable-swap-and-transparent-huge-pages\">Should I disable swap and Transparent Huge Pages?<\/h3>\t\t\t\t<div>\n\t\t\t\t\t\t<div class=\"sc_fs_faq__content\">\n\t\t\t\t\n\n<p>Yes. Swap and THP introduce latency and unpredictability under load. Disable swap, set swappiness low (1), disable THP, and enable bootstrap.memory_lock to keep Elasticsearch memory resident.<\/p>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section\t\thelp class=\"sc_fs_faq sc_card    \"\n\t\t\t\t>\n\t\t\t\t<h3 id=\"how-do-i-speed-up-bulk-indexing\">How do I speed up bulk indexing?<\/h3>\t\t\t\t<div>\n\t\t\t\t\t\t<div class=\"sc_fs_faq__content\">\n\t\t\t\t\n\n<p>Temporarily set replicas=0 and refresh_interval to 30s or -1, then use the _bulk API with 5\u201315 MB batches and parallel workers. Re-enable replicas and normal refresh afterward. Auto-generated IDs also help.<\/p>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section\t\thelp class=\"sc_fs_faq sc_card    \"\n\t\t\t\t>\n\t\t\t\t<h3 id=\"what-linux-settings-matter-most-for-elasticsearch-performance\">What Linux settings matter most for Elasticsearch performance?<\/h3>\t\t\t\t<div>\n\t\t\t\t\t\t<div class=\"sc_fs_faq__content\">\n\t\t\t\t\n\n<p>Set vm.max_map_count=262144, increase file descriptors, disable swap and THP, and use SSD\/NVMe with noatime. Keep the kernel and filesystem tuned, and monitor I\/O wait, GC, and shard counts consistently.<\/p>\n\n\n\n<p>With these steps, you can confidently optimize Elasticsearch on a Linux server for both indexing throughput and low-latency search. If you\u2019d rather focus on your app, YouStable can architect, deploy, and manage a fully optimized Elasticsearch stack tailored to your workload and budget.<\/p>\n\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t<\/section>\n\t\t\n<script type=\"application\/ld+json\">\n\t{\n\t\t\"@context\": \"https:\/\/schema.org\",\n\t\t\"@type\": \"FAQPage\",\n\t\t\"mainEntity\": [\n\t\t\t\t\t{\n\t\t\t\t\"@type\": \"Question\",\n\t\t\t\t\"name\": \"What is the best JVM heap size for Elasticsearch?\",\n\t\t\t\t\"acceptedAnswer\": {\n\t\t\t\t\t\"@type\": \"Answer\",\n\t\t\t\t\t\"text\": \"<p>Set Xms and Xmx to the same value, about 50% of system RAM, capped at 31 GB. This preserves compressed object pointers and keeps GC efficient. Leave the remaining RAM for the OS cache and Lucene.<\/p>\"\n\t\t\t\t\t\t\t\t\t}\n\t\t\t}\n\t\t\t,\t\t\t\t{\n\t\t\t\t\"@type\": \"Question\",\n\t\t\t\t\"name\": \"How many shards should my index have?\",\n\t\t\t\t\"acceptedAnswer\": {\n\t\t\t\t\t\"@type\": \"Answer\",\n\t\t\t\t\t\"text\": \"<p>Aim for shards in the 20\u201350 GB range. Use enough primaries to distribute load across data nodes but avoid creating too many small shards. ILM with rollover by size\/time helps maintain healthy shard sizes automatically.<\/p>\"\n\t\t\t\t\t\t\t\t\t}\n\t\t\t}\n\t\t\t,\t\t\t\t{\n\t\t\t\t\"@type\": \"Question\",\n\t\t\t\t\"name\": \"Should I disable swap and Transparent Huge Pages?\",\n\t\t\t\t\"acceptedAnswer\": {\n\t\t\t\t\t\"@type\": \"Answer\",\n\t\t\t\t\t\"text\": \"<p>Yes. Swap and THP introduce latency and unpredictability under load. Disable swap, set swappiness low (1), disable THP, and enable bootstrap.memory_lock to keep Elasticsearch memory resident.<\/p>\"\n\t\t\t\t\t\t\t\t\t}\n\t\t\t}\n\t\t\t,\t\t\t\t{\n\t\t\t\t\"@type\": \"Question\",\n\t\t\t\t\"name\": \"How do I speed up bulk indexing?\",\n\t\t\t\t\"acceptedAnswer\": {\n\t\t\t\t\t\"@type\": \"Answer\",\n\t\t\t\t\t\"text\": \"<p>Temporarily set replicas=0 and refresh_interval to 30s or -1, then use the _bulk API with 5\u201315 MB batches and parallel workers. Re-enable replicas and normal refresh afterward. Auto-generated IDs also help.<\/p>\"\n\t\t\t\t\t\t\t\t\t}\n\t\t\t}\n\t\t\t,\t\t\t\t{\n\t\t\t\t\"@type\": \"Question\",\n\t\t\t\t\"name\": \"What Linux settings matter most for Elasticsearch performance?\",\n\t\t\t\t\"acceptedAnswer\": {\n\t\t\t\t\t\"@type\": \"Answer\",\n\t\t\t\t\t\"text\": \"<p>Set vm.max_map_count=262144, increase file descriptors, disable swap and THP, and use SSD\/NVMe with noatime. Keep the kernel and filesystem tuned, and monitor I\/O wait, GC, and shard counts consistently.<\/p><p>With these steps, you can confidently optimize Elasticsearch on a Linux server for both indexing throughput and low-latency search. If you\u2019d rather focus on your app, YouStable can architect, deploy, and manage a fully optimized Elasticsearch stack tailored to your workload and budget.<\/p>\"\n\t\t\t\t\t\t\t\t\t}\n\t\t\t}\n\t\t\t\t\t\t]\n\t}\n<\/script>\n\n\n\n<h2 class=\"wp-block-heading\" class=\"wp-block-heading\" id=\"conclusion\">Conclusion<\/h2>\n\n\n\n<p>Optimizing Elasticsearch on <a href=\"https:\/\/www.youstable.com\/blog\/optimize-redis-on-linux\/\">Linux starts<\/a> with correct JVM heap sizing, disabled or minimized swapping, and sufficient file descriptors so memory and IO are predictable under load. <\/p>\n\n\n\n<p>Combine that foundation with <a href=\"https:\/\/www.youstable.com\/blog\/ssd-dedicated-server-vs-hdd-dedicated\/\">fast SSD storage<\/a>, balanced shard and index design, and bulk-friendly indexing patterns to prevent hotspots and saturation. Finally, design queries and mappings carefully, monitor key metrics (heap, GC, IO, shard count), and iterate configuration as data and traffic grow to sustain performance.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>To optimize Elasticsearch on Linux server, allocate the JVM heap to ~50% of RAM (max 31 GB), use SSD\/NVMe storage, [&hellip;]<\/p>\n","protected":false},"author":13,"featured_media":14072,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"iawp_total_views":3,"footnotes":""},"categories":[350],"tags":[2141,2160],"class_list":["post-13741","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-knowledgebase","tag-linux-server","tag-optimize-elasticsearch-on-linux"],"acf":[],"featured_image_src":"https:\/\/www.youstable.com\/blog\/wp-content\/uploads\/2025\/12\/How-to-Optimize-ElasticSearch-on-Linux-Server.jpg","author_info":{"display_name":"Prahlad Prajapati","author_link":"https:\/\/www.youstable.com\/blog\/author\/prahladblog"},"_links":{"self":[{"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/posts\/13741","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/comments?post=13741"}],"version-history":[{"count":3,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/posts\/13741\/revisions"}],"predecessor-version":[{"id":14015,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/posts\/13741\/revisions\/14015"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/media\/14072"}],"wp:attachment":[{"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/media?parent=13741"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/categories?post=13741"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.youstable.com\/blog\/wp-json\/wp\/v2\/tags?post=13741"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}