Amazon OpenSearch Service (formerly known as Amazon Elasticsearch Service) is a fully managed service designed to help users search, analyze, and visualize data in real-time. Its robust architecture and extensive integrations with AWS services make it a powerful tool for both operational and analytical applications, supporting a wide range of use cases including log analytics, application monitoring, and security analytics. This chapter provides an overview of the key components, applications, security, and storage features available in Amazon OpenSearch Service.
Amazon OpenSearch Service enables powerful search and analytics capabilities, supporting diverse applications:
- Full-text Search: Enhances application search capabilities to deliver fast, accurate, and relevant search results for user queries.
- Log Analytics: Aggregates, monitors, and analyzes log files for improved operational insights and troubleshooting.
- Application Monitoring: Tracks application performance and health metrics in real-time for proactive optimization.
- Security Analytics: Analyzes security data to detect threats and vulnerabilities, supporting defensive strategies.
- Clickstream Analytics: Examines web and application traffic data to understand user behavior and optimize user experiences.
Amazon OpenSearch Service includes several core components that enable the creation and management of search and analytics workloads:
- Domains: A domain represents a managed OpenSearch cluster, encapsulating the setup and configuration of an OpenSearch deployment.
- Documents: The basic unit of information that can be indexed, structured as JSON objects. Each document contains fields as key-value pairs.
- Indices: Collections of documents that serve a similar purpose. An index is a logical partition that organizes and provides fast access to data.
- Shards: Partitions of an index, distributed across nodes for efficient storage and performance. Shards can be primary (original data) or replicas (copies for redundancy).
- Nodes: Single instances within a cluster that store data and perform indexing and searching. A typical setup includes three master nodes to ensure resilience.
Tip: If memory issues arise, consider reducing the number of shards by deleting old indices.
Amazon OpenSearch Service incorporates several security features to protect data and control access effectively:
- Resource-based Policies: Define and attach policies directly to OpenSearch domains to manage access at the resource level.
- Identity-based Policies: Use AWS IAM to set permissions for AWS users and roles, controlling actions on OpenSearch resources.
- IP-based Policies: Restrict domain access based on IP addresses, allowing only trusted networks to interact with your domains.
- Request Signing: Use AWS Signature Version 4 to authenticate requests securely without exposing sensitive credentials.
- VPC Support: Deploy OpenSearch domains within a VPC for network isolation and leverage security groups and ACLs for fine-grained control.
- Integration with Amazon Cognito: Manage user access to OpenSearch dashboards through Amazon Cognito, enabling sign-in, sign-up, and access control.
Amazon OpenSearch Service provides various storage options to support a range of operational needs:
- Standard Data Nodes (Hot Storage): Use "hot" storage for high-throughput, performance-sensitive workloads. These nodes use EBS storage for fast access.
- UltraWarm Storage: A cost-effective storage tier integrated with Amazon S3, ideal for storing less frequently accessed data, such as indices with few write operations.
- Cold Storage: The most cost-effective option for rarely accessed data, suitable for use cases like forensic analysis or periodic research on older datasets, also stored in S3.
Amazon OpenSearch Service is a powerful, flexible platform for real-time data analysis, search, and visualization, with extensive security and storage options to suit a wide range of application requirements.