Posted inUS_1

MinIO Client (mc) for Server Management: A Comprehensive Guide

No Comments

The MinIO Client, known as mc, is a versatile command-line tool designed to interact with file systems and Amazon S3-compatible cloud storage services. It serves as a modern alternative to traditional UNIX commands, offering functionalities like ls, cat, cp, mirror, and diff, but extended to support cloud environments. This makes mc an indispensable tool for managing your storage infrastructure, including listing servers and buckets, and manipulating objects within them.

mc is specifically built for seamless compatibility with the AWS S3 API and rigorously tested with both MinIO and AWS S3 to ensure consistent and reliable performance. While it may function with other S3-compatible services, MinIO does not guarantee functionality beyond these officially supported platforms. Using mc with unsupported S3 services is at your own discretion and risk.

The general syntax for mc commands is as follows:

mc[GLOBALFLAGS]COMMAND --help

For a quick overview of available commands, refer to the Command Quick Reference section.

Version Compatibility with MinIO Server

MinIO Client and MinIO Server are released independently. To ensure optimal performance and avoid potential compatibility issues, it’s recommended to use a MinIO Client version that is released around the same time as or later than your MinIO Server version.

While using a more recent MinIO Client with an older MinIO Server is generally acceptable, significant version discrepancies may lead to warnings or errors. Core S3 API functionalities, such as copying operations (mc cp), are likely to remain consistent. However, newer features or command flags might only be fully supported and stable when the client and server versions are closely aligned.

Quick Start Guide to Using MinIO Client for Server Management

Let’s get you started with the MinIO Client. The following steps will guide you through installation, setting up an alias to your server, and testing the connection.

Step 1: Installing mc

To begin, you need to install the mc command-line tool on your local machine. Select the tab corresponding to your operating system or environment for specific installation instructions.

The commands provided below will temporarily add mc to your system’s PATH environment variable. For permanent modifications, please consult your operating system’s documentation. Alternatively, you can execute mc by navigating to its installation directory and running ./mc --help.

64-bit Intel (Linux)

curl https://dl.min.io/client/mc/release/linux-amd64/mc 
  --create-dirs 
  -o $HOME/minio-binaries/mc
chmod +x $HOME/minio-binaries/mc
export PATH=$PATH:$HOME/minio-binaries/
mc --help

64-bit PPC (Linux)

curl https://dl.min.io/client/mc/release/linux-ppc64le/mc 
  --create-dirs 
  -o ~/minio-binaries/mc
chmod +x $HOME/minio-binaries/mc
export PATH=$PATH:$HOME/minio-binaries/
mc --help

ARM64 (Linux)

curl https://dl.min.io/client/mc/release/linux-arm64/mc 
  --create-dirs 
  -o ~/minio-binaries/mc
chmod +x $HOME/minio-binaries/mc
export PATH=$PATH:$HOME/minio-binaries/
mc --help

Installation via MinIO Download Page (Linux)

MinIO binaries are officially distributed through the MinIO Download Page. While not available in standard Linux repositories or package managers (like Ubuntu, RHEL, or Archlinux/AUR), using the official download ensures you get the correct and unaltered package.

Installing mc from package managers is discouraged as they might offer outdated or renamed versions. All documentation assumes the official mc client binary downloaded directly from the MinIO Download Page.

macOS

brew install minio/stable/mc
mc --help

Windows

Open the following link in your web browser to download the mc.exe executable:

https://dl.min.io/client/mc/release/windows-amd64/mc.exe

Execute the downloaded file by double-clicking it or by running the following command in your command prompt or PowerShell, replacing pathto with the actual path to the mc.exe file:

pathtomc.exe --help

Installation from Source (Advanced Users)

Installing from source is intended for developers and advanced users and requires a working Go programming environment. Refer to the official Go installation guide for instructions.

To install mc from source, execute the following command in your terminal:

go install github.com/minio/mc@latest

Note that mc update command is not supported for source-based installations.

Step 2: Setting up an Alias for Your S3-Compatible Service

Setting up an alias is crucial for easily managing your server connections with mc.

Important Security Note: The following example temporarily disables bash history to prevent accidental exposure of authentication credentials in plain text. This is a basic security measure and may not cover all potential security risks. Always adhere to your operating system’s security best practices when entering sensitive information via the command line.

Use the mc alias set command to configure a connection to your Amazon S3-compatible service within mc. This configuration is stored for future use.

bash +o history
mc alias set ALIAS HOSTNAME ACCESS_KEY SECRET_KEY
bash -o history

Replace the placeholders with your actual server details:

  • ALIAS: Choose a unique name to identify this S3 service. You’ll use this alias in subsequent mc commands to specify the target server.
  • HOSTNAME: Enter the URL endpoint or IP address of your S3 service.
  • ACCESS_KEY and SECRET_KEY: Provide the access key and secret key for a user account on your S3 service.

If you omit the ACCESS_KEY and SECRET_KEY, mc will prompt you to enter them securely in the command line.

Here are provider-specific examples for setting up aliases:

MinIO Server:

mc alias set myminio https://minioserver.example.net ACCESS_KEY SECRET_KEY

Amazon S3:

mc alias set myS3 https://s3.{your-region-code}.amazonaws.com ACCESS_KEY SECRET_KEY

Google Cloud Storage (GCS):

mc alias set myGCS https://storage.googleapis.com ACCESS_KEY SECRET_KEY

Step 3: Testing the Server Connection

After setting up the alias, it’s essential to verify the connection to your S3 service. Use the mc admin info command for this purpose.

mc admin info myminio

If the connection is successful, the command will return information about your S3 service. If it fails, check the following:

  • Network Connectivity: Ensure your machine can reach the S3 service URL. Use tools like ping or traceroute to verify network connectivity.
  • Credentials: Double-check that the ACCESS_KEY and SECRET_KEY are correct and associated with a user on the S3 service who has the necessary permissions to perform actions.

For MinIO server deployments, consult the MinIO Access Management documentation for details on user permissions. For other S3-compatible services, refer to their respective documentation.

Command Quick Reference

The following table provides a summary of commonly used mc commands. These commands empower you to efficiently manage your servers and data.

Note: The MinIO Client also includes administrative commands specifically for managing MinIO deployments. These are documented separately under mc admin. The table below excludes these administrative commands and focuses on general-purpose object storage management commands.

Command Description
mc alias list, mc alias remove, mc alias set, mc alias import, mc alias export Manage connection aliases for S3-compatible hosts, making server management easier and more organized.
mc anonymous get, mc anonymous get-json, mc anonymous links, mc anonymous list, mc anonymous set, mc anonymous set-json Control anonymous access policies for buckets and their contents, enabling public access for specific actions without authentication.
mc batch describe, mc batch generate, mc batch list, mc batch start, mc batch status Execute batch operations on a MinIO deployment, allowing for efficient management of multiple tasks.
mc cat Concatenate and display file or object content, similar to the UNIX cat command, useful for quick content inspection.
mc cp Copy objects between MinIO deployments, local filesystems, and other S3-compatible services, providing flexible data transfer options.
mc diff Compare differences between filesystem directories or MinIO buckets, identifying missing or size-differing objects without comparing content, useful for synchronization checks.
mc du Summarize disk usage for buckets and folders, similar to the UNIX du command, helping you monitor storage consumption.
mc encrypt clear, mc encrypt info, mc encrypt set Manage default bucket Server-Side Encryption (SSE) modes, ensuring data at rest is automatically encrypted for security.
mc event add, mc event ls, mc event rm Configure, list, and remove bucket event notifications, enabling event-driven workflows and monitoring of bucket activities.
mc find Search for objects within MinIO deployments or files within local filesystems based on various criteria, facilitating efficient data discovery.
mc get Download objects from S3 deployments to your local filesystem, retrieving data from your cloud storage.
mc head Display the first N lines of an object, allowing for quick inspection of object headers or the beginning of content.
mc idp ldap ..., mc idp openid ... Manage integrations with external Identity Providers (LDAP and OpenID), streamlining user authentication and access management.
mc ilm restore, mc ilm rule ..., mc ilm tier ... Manage object lifecycle management (ILM) rules and tiering, automating object transitions and optimizing storage costs based on data access patterns.
mc legalhold clear, mc legalhold info, mc legalhold set Manage object legal hold (WORM) settings, ensuring data immutability for compliance and regulatory requirements.
mc license info, mc license register, mc license update Manage MinIO SUBNET licensing, including registration, information retrieval, and license updates.
mc ls List buckets and objects on MinIO or other S3-compatible services, providing a fundamental command for storage exploration and server listing.
mc mb Create new buckets or directories, setting up storage containers for your data.
mc mirror Synchronize content between filesystems, MinIO deployments, and S3-compatible hosts, similar to rsync, ensuring data consistency across locations.
mc mv Move objects between locations within MinIO or between MinIO and local filesystems, reorganizing data efficiently.
mc od Upload large local files to remote locations in parts, optimizing upload speed and handling large data transfers.
mc ping Perform a liveness check on a specified target, verifying server availability and responsiveness.
mc pipe Stream content from STDIN to a target object, enabling data ingestion from standard input.
mc put Upload objects from the local filesystem to a bucket on an S3 deployment, transferring local data to cloud storage.
mc rb Remove buckets on MinIO or other S3-compatible services. Use mc rm to remove bucket contents only.
mc ready Check the status of a MinIO cluster and its read/write quorum, ensuring cluster health and operational readiness.
mc replicate ... Configure and manage server-side bucket replication, including active-active setups and resynchronization, ensuring data redundancy and high availability.
mc retention clear, mc retention info, mc retention set Configure Write-Once Read-Many (WORM) locking settings for objects and buckets, enforcing data retention policies for compliance.
mc rm Remove objects from a bucket, managing object deletion within your storage environment. Use mc rb to remove the entire bucket.
mc share download, mc share ls, mc share upload Manage presigned URLs for secure object download and upload, enabling controlled access to your data.
mc sql Perform SQL queries on objects within MinIO deployments using S3 Select interface, enabling data analysis directly within storage.
mc stat Display object and bucket metadata, providing detailed information about your storage resources.
mc support ... Access support tools for analyzing deployment health and performance, running diagnostics, and uploading reports for MinIO engineering analysis.
mc tag list, mc tag remove, mc tag set Manage tags associated with buckets and objects, enabling metadata management and categorization.
mc tree List prefixes within a MinIO bucket in a tree format, visualizing bucket structure and optionally listing objects within each prefix.
mc undo Reverse accidental PUT or DELETE operations at a specified path, recovering from unintended data modifications.
mc update Automatically update the mc binary to the latest stable version, ensuring you have the most current features and fixes.
mc version enable, mc version info, mc version suspend Manage bucket versioning, enabling, disabling, and checking versioning status for data protection and recovery.
mc watch Monitor events on MinIO buckets or local filesystem paths, enabling real-time observation of storage activities. For S3 services, use mc event add for event notifications.

Configuration File

mc utilizes a JSON formatted configuration file to store settings, including the aliases you define for your S3-compatible services.

On Linux and macOS systems, the default configuration file is located at ~/.mc/config.json.

For Windows, mc attempts to locate the configuration file by checking the following environment variables in order:

  1. HOME.mcconfig.json
  2. USERPROFILE.mcconfig.json
  3. HOMEDRIVE+HOMEPATH.mcconfig.json

If none of these paths are valid, mc will return an error. You can also specify a custom configuration directory using the --config-dir global option.

Certificates

MinIO Client stores certificates and Certificate Authorities (CAs) for secure connections. The default storage paths are:

Linux, macOS, and Unix-like systems:

~/.mc/certs/         # Certificates
~/.mc/certs/CAs/      # Certificate Authorities

Windows systems:

C:Users[username]mccerts      # Certificates
C:Users[username]mccertsCAs   # Certificate Authorities

When you create a new alias, mc fetches the server’s certificate, calculates its public key fingerprint, and prompts you to confirm whether to trust the certificate. If you choose to trust it, mc saves the certificate to the appropriate certificate authority path.

Note: For testing purposes in non-production environments, you can bypass certificate verification for specific mc commands by using the --insecure flag. However, exercise caution when using this option with untrusted S3 hosts.

Pattern Matching

Several mc commands and flags support pattern matching using wildcards for flexible object selection. The following wildcards are supported:

  • *: Matches any string of characters within a path or filename.
  • ?: Matches any single character.

Refer to the table below for examples of wildcard usage and their matching behavior.

Pattern Text Match Result
abc* ab Match
abc* abd Not a match
abc*c abcd Match
ab*??d abxxc Match
ab*??d abxc Match
ab??d abxc Match
ab??d abc Match
ab??d abcxdd Not a match

Global Options

mc commands support various global options to modify their behavior. You can also set some of these options using Environment Variables.

  • --config-dir: Specifies the path to the JSON configuration file. Alternatively, use the MC_CONFIG_DIR environment variable. See Configuration File for details.

  • --debug: Enables verbose output to the console for detailed debugging information. Alternatively, use the MC_DEBUG environment variable.

    mc --debug ls play

  • --disable-pager, --dp: Disables the pager functionality, printing raw output to STDOUT. (New in version mc: RELEASE.2024-04-29T09-56-05Z)

  • --insecure: Disables TLS/SSL certificate verification, allowing connections to servers with invalid certificates. Use with caution, especially with untrusted hosts. Alternatively, use the MC_INSECURE environment variable.

  • --json: Enables JSON Lines formatted output to the console for structured data. Alternatively, use the MC_JSON environment variable.

    mc --json ls play

  • --no-color: Disables colored console output, useful for dumb terminals. Alternatively, use the MC_NO_COLOR environment variable.

  • --quiet: Suppresses all console output except errors. Alternatively, use the MC_QUIET environment variable.

  • --resolve: Creates custom DNS mappings to resolve a hostname to a specific IP address. (New in version mc: RELEASE.2024-08-13T05-33-17Z)

    Syntax: --resolve HOST[:PORT]=IP

    Example:

    mc alias set --resolve myminio.example.com:9000=192.168.188.118 'myminio' 'https://myminio.example.com:9000' 'miniouser' 'miniosecret'

    You can use this flag multiple times for different mappings.

  • --version: Displays the current version of mc.

  • --help: Displays command usage instructions on the terminal.

This comprehensive guide provides you with the necessary information to effectively use the MinIO Client (mc) for managing your servers and object storage. From basic commands like listing servers and buckets to advanced features like replication and lifecycle management, mc offers a powerful and efficient way to interact with your S3-compatible storage infrastructure.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *