abacus-cf-applications
CF app usage reporting bridge.
The bridge reports the metrics defined in linux-container resource:
- instance_memory [GB]
- running_instances [number]
For every app usage event from CF the bridge POSTs time-based usage report as follows:
- for STOPPED event: reports 0 memory and 0 insances since the timestamp in the STOPPED event
- for other events: reports the actual memory and number of instances since the timestamp of the event
UAA Clients
The applications bridge communicates with Cloud Controller. Register cf-applications application as CF client with:
gem install cf-uaacuaac target uaa.bosh-lite.com --skip-ssl-validationuaac token client get admin -s admin-secretuaac client add abacus-cf-applications --name abacus-cf-applications --authorized_grant_types client_credentials --authorities cloud_controller.admin --secret secret
If you use secured Abacus installation you will need an additional resource client:
uaac client add abacus-linux-container --name abacus-linux-container --authorized_grant_types client_credentials --authorities abacus.usage.linux-container.write,abacus.usage.linux-container.read --scope abacus.usage.linux-container.write,abacus.usage.linux-container.read --secret secret
Note: Take care to set change the client ID and secret in the examples above.
Start the applications bridge
Locally
The steps below use Abacus running locally. We also assume that the Abacus installation has the abacus-authentication-plugin as a token provider.
To start the applications bridge locally against CF running on BOSH Lite set the API address:
export API=https://api.bosh-lite.com
Set the used client ID and secret with:
export CF_CLIENT_ID=abacus-cf-applicationsexport CF_CLIENT_SECRET=secret
In case of secured Abacus set the JWT algorithm, key and the resource provider client credentials:
export CLIENT_ID=abacus-linux-containerexport CLIENT_SECRET=secretexport JWTKEY=secretexport JWTALGO=HS256
You can optionally enable the debug output with:
export DEBUG=abacus-cf-*
Finally start the applications bridge with:
cd ~/workspace/cf-abacusyarn start cf
To stop the applications bridge:
yarn stop cf
Cloud Foundry
To start the applications bridge on CF running on BOSH Lite follow the steps below.
Setup CF:
./bin/cfsetup
Go to applications bridge directory:
cd ~/workspace/cf-abacus/lib/cf/applications
Edit the manifest.yml
to look like this:
applications:- name: abacus-cf-applications host: abacus-cf-applications path: . instances: 1 memory: 512M disk_quota: 512M env: CONF: default DEBUG: e-abacus*,abacus-cf* COLLECTOR: abacus-usage-collector DB: mongodb://localhost:27017 EUREKA: abacus-eureka-plugin API: https://api.bosh-lite.com:443 AUTH_SERVER: https://api.bosh-lite.com:443 CF_CLIENT_ID: abacus-cf-applications CF_CLIENT_SECRET: secret
In case you are running a secured Abacus installation, add the following entries:
SECURED: true CLIENT_ID: abacus-linux-container CLIENT_SECRET: secret JWTKEY: |+ -----BEGIN PUBLIC KEY----- ... <UAA public key in PEM format> ... -----END PUBLIC KEY----- JWTALGO: RS256
Build, pack and push the applications bridge to Cloud Foundry:
yarn install && yarn run lint && yarn test &&&& yarn run cfpush
Create a database service instance, called db
and bind it to abacus-cf-applications
:
cf create-service mongodb-3.0.7-lite free dbcf bind-service abacus-cf-applications db
In case you want to use external DB you can do this by adding DB
to the deployment manifest:
DB: mongodb://user:password@mymongohost.com:27017/databaseName?ssl=true
Start the applications bridge:
cf start abacus-cf-applications
Tail the logs to check the progress:
cf logs abacus-cf-applications
You can change the client ID and secret used to communicate with CC like so:
cf set-env abacus-cf-applications CLIENT_ID <client_id>
cf set-env abacus-cf-applications CLIENT_SECRET <secret>
cf restart abacus-cf-applications
To change the resource provider (abacus-linux-container) settings or the number of connections, set the respective environment variables using cf set-env
.
Configuration
Bridge internal timeouts can be configured by modifying these environment variables:
- MIN_INTERVAL_TIME
- minimum time [milliseconds] between each call to CF app usage events API
- this variable also controls the time between each attempt to cache the last processd app usage GUID. The bridge tries to cache the GUID every 5 * MIN_INTERVAL_TIME milliseconds
- MAX_INTERVAL_TIME
- maximum time [milliseconds] between app usage calls to CF
- maximum time between Abacus reporting attempts
- GUID_MIN_AGE - determines how old an app usage events should be to be reported to Abacus. New events order is not guaranteed in CC database. That's why we store only events older than the GUID_MIN_AGE.
Note: The timeout between CF API calls and Abacus usage retries is increased exponentially with each failed attempt.
Statistics
The applications bridge exposes the /v1/stats/
endpoint that provides performance metrics and call statistics. A snippet of the values returned:
"cache": , "statistics":
The following data is available:
Cache content:
- lastRecordedGUID: GUID of the last reported event
- lastRecordedTimestamp: Timestamp of the last reported event. For example 2015-08-18T11:28:20Z
Operation statistics:
- cache.read/write: Number of cache operations. The cache stores the last processed app usage event GUID.
- usage
- missingToken: Missing abacus resource token
- reportFailures: Number of failed usage reports
- reportSuccess: Successful usage reports
- loopFailures: Number of report loop failures
- loopSuccess: Number of successful report loop cycles
- paging
- missingToken: Missing CF CC token
- pageReadSuccess: Number of successful page reads
- pageReadFailures: Failed page reads
- pageProcessSuccess: Number of successfully processed resources
- pageProcessFailures: Number of unsuccessfully processed resources
- pageProcessEnd: Number of processed pages