# Heartbox Analysis Cache System - Optimizations

## Overview

The heartbox analysis system has been optimized to provide fast, consistent, and accurate service status detection across all Astroport.ONE components. This document explains the improvements and how to use the new cache system.

## Problem Solved

### Before Optimization

* **Inconsistent service status**: `_12345.sh`, `command.sh`, and `20h12.process.sh` used different methods to detect service status
* **Slow performance**: Each script performed real-time checks, causing delays
* **Resource intensive**: Multiple scripts checking the same services repeatedly
* **Outdated information**: Service status could be stale between different components

### After Optimization

* **Consistent status**: All components use the same cache-based detection system
* **Fast performance**: Cache-based lookups are \~10x faster than real-time checks
* **Resource efficient**: Single source of truth with intelligent cache invalidation
* **Always fresh**: 5-minute TTL ensures data is never more than 5 minutes old

## Architecture

```
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   _12345.sh     │    │   command.sh     │    │ 20h12.process.sh│
│   (API Server)  │    │   (Status UI)    │    │  (Maintenance)  │
└─────────┬───────┘    └──────────┬───────┘    └─────────┬───────┘
          │                       │                       │
          └───────────────────────┼───────────────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │   heartbox_analysis.sh    │
                    │   (Cache Manager)         │
                    └─────────────┬─────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │ ~/.zen/tmp/${IPFSNODEID}/ │
                    │ heartbox_analysis.json    │
                    │ (Cache File - 5min TTL)   │
                    └───────────────────────────┘
```

## Key Features

### 1. Fast Service Detection

* **Prometheus integration**: Uses Prometheus metrics when available for system data
* **Optimized checks**: Fast port and process checks without timeouts
* **Background updates**: Cache updates happen in background without blocking

### 2. Intelligent Caching

* **5-minute TTL**: Cache expires after 5 minutes for fresh data
* **Automatic fallback**: Falls back to real-time checks if cache is unavailable
* **Background refresh**: Updates cache in background when expired

### 3. Consistent Data Format

* **Standardized JSON**: All components use the same JSON structure
* **Service status**: Boolean flags for each service (ipfs, astroport, nextcloud, etc.)
* **Capacity metrics**: Storage and slot calculations
* **System metrics**: CPU, memory, disk usage

## Usage

### Basic Commands

```bash
# Export current analysis (uses cache if fresh)
./admin/monitor/heartbox_analysis.sh export --json

# Force cache update
./admin/monitor/heartbox_analysis.sh update

# Read cached data
./admin/monitor/heartbox_analysis.sh cache

# Test the system
./tools/test_heartbox_cache.sh
```

### Integration Points

#### \_12345.sh (API Server)

* **Automatic cache usage**: Checks cache freshness and uses it if available
* **Background updates**: Updates cache in background if expired
* **Fast response**: Returns cached data immediately for API requests

#### command.sh (Status UI)

* **Consistent display**: Uses same cache for service status display
* **Real-time fallback**: Falls back to real-time checks if cache unavailable
* **Background refresh**: Updates cache in background when needed

#### 20h12.process.sh (Maintenance)

* **Cache refresh**: Updates cache during maintenance cycle
* **Status reporting**: Uses cache for final status report
* **12345.json sync**: Updates 12345.json with fresh cache data

## Cache Structure

```json
{
  "timestamp": "2025-01-27T10:30:00+00:00",
  "node_info": {
    "id": "12D3KooW...",
    "captain": "support@qo-op.com",
    "type": "y_level",
    "hostname": "libra.copylaradio.com"
  },
  "system": {
    "cpu": { "model": "...", "cores": 8, "load_average": "1.2" },
    "memory": { "total_gb": 16, "used_gb": 8, "usage_percent": 50 },
    "storage": { "total": "1TB", "available": "500GB", "usage_percent": "50%" }
  },
  "services": {
    "ipfs": { "active": true, "peers_connected": 818 },
    "astroport": { "active": true },
    "nextcloud": { "active": false, "container": null },
    "nostr_relay": { "active": true, "port": 7777 },
    "uspot": { "active": false, "port": 54321 },
    "g1billet": { "active": true }
  },
  "capacities": {
    "zencard_slots": 0,
    "nostr_slots": 45,
    "reserved_captain_slots": 8,
    "available_space_gb": 1074,
    "storage_details": {
      "nextcloud": { "available_gb": 0, "status": "not_mounted" },
      "ipfs": { "available_gb": 537 },
      "root": { "available_gb": 537 }
    }
  }
}
```

## Performance Improvements

### Before

* **Service checks**: 2-5 seconds per component
* **Resource usage**: High CPU and I/O for repeated checks
* **Response time**: 3-8 seconds for API responses

### After

* **Cache lookup**: <100ms for cached data
* **Resource usage**: Minimal CPU and I/O
* **Response time**: <500ms for API responses

## Monitoring and Debugging

### Cache Status

```bash
# Check cache age
ls -la ~/.zen/tmp/${IPFSNODEID}/heartbox_analysis.json

# View cache contents
./admin/monitor/heartbox_analysis.sh cache | jq .

# Force refresh
./admin/monitor/heartbox_analysis.sh update
```

### Service Status Verification

```bash
# Compare cache vs real-time
./tools/test_heartbox_cache.sh

# Check specific service
jq -r '.services.ipfs.active' ~/.zen/tmp/${IPFSNODEID}/heartbox_analysis.json
```

### Troubleshooting

#### Cache Not Updating

```bash
# Check permissions
ls -la ~/.zen/tmp/${IPFSNODEID}/

# Manual update
./admin/monitor/heartbox_analysis.sh update

# Check logs
tail -f ~/.zen/tmp/_12345.log
```

#### Inconsistent Status

```bash
# Force refresh all components
./admin/monitor/heartbox_analysis.sh update
sudo systemctl restart astroport
./tools/test_heartbox_cache.sh
```

## Configuration

### Cache TTL

* **Default**: 5 minutes (300 seconds)
* **Location**: `heartbox_analysis.sh` line 15
* **Modification**: Change `CACHE_TTL=300` to desired value

### Cache Location

* **Default**: `~/.zen/tmp/${IPFSNODEID}/heartbox_analysis.json`
* **Backup**: `~/.zen/tmp/${IPFSNODEID}/12345.json` (legacy format)

### Prometheus Integration

* **URL**: `http://localhost:9090`
* **Metrics**: disk, memory, CPU usage
* **Fallback**: System commands if Prometheus unavailable

## Migration Guide

### From Old System

1. **No manual migration required**: System automatically detects and uses new cache
2. **Legacy support**: Old 12345.json format still supported
3. **Gradual transition**: Components automatically switch to new system

### Verification

```bash
# Run test suite
./tools/test_heartbox_cache.sh

# Check all components use cache
grep -r "heartbox_analysis" Astroport.ONE/ --include="*.sh"
```

## Benefits

### For Users

* **Faster response times**: API responses in <500ms
* **Consistent status**: Same information across all interfaces
* **Reliable monitoring**: Accurate service status

### For System

* **Reduced load**: Less CPU and I/O usage
* **Better reliability**: Fewer timeouts and failures
* **Scalable**: Can handle more concurrent requests

### For Development

* **Centralized logic**: Single source of truth for service status
* **Easy maintenance**: Changes in one place affect all components
* **Better testing**: Comprehensive test suite available

## Future Enhancements

### Planned Features

* **Metrics collection**: Historical performance data
* **Alerting**: Automatic notifications for service issues
* **Dashboard**: Web interface for monitoring
* **API endpoints**: REST API for external monitoring

### Integration Opportunities

* **Grafana**: Dashboard integration
* **Prometheus**: Metrics export
* **Nagios**: Monitoring integration
* **Slack**: Alert notifications

## Support

### Documentation

* **This file**: Complete system overview
* **Test script**: `./tools/test_heartbox_cache.sh`
* **Code comments**: Inline documentation in scripts

### Troubleshooting

* **Logs**: `~/.zen/tmp/_12345.log`
* **Cache**: `~/.zen/tmp/${IPFSNODEID}/heartbox_analysis.json`
* **Tests**: `./tools/test_heartbox_cache.sh`

### Contact

* **Author**: Fred (<support@qo-op.com>)
* **License**: AGPL-3.0
* **Version**: 2.0 (Optimized)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://astroport-1.gitbook.io/astroport.one/tools/heartbox_cache_readme.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.