Android & Elastic Beats

Android & ElasticBeats

Introduction

This blog aims to explore how one can build a lightweight endpoint sensor for mobile capable of collecting device meta-data, forensics, and potential indicators of compromise using ELK stack. Most endpoint management solutions have some kind of backend that processes data from agent apps. Engineering spends a considerable amount of time building, maintaining, optimizing numerous backend services to make sense of this data.

OSquery, even though it is for desktop/server platform, alleviated this problem by allowing administrators to query a device using SQL-like statements. This lead to countless security products which applied querying, merging & aggregating techniques on endpoint data. But OSquery is not available for mobile, necessitating the need to architect such a system for mobile (Android/iOS).

In this article, I will try to build such a system using ELK and a lightweight endpoint sensor based on ElasticBeats. ElasticBeats is designed to collect and transport data from a given source (server/desktop/container/pod) to ElasticSearch. There are “pre-made” beats for different use-cases. E.g. PacketBeat for sending network traffic, FileBeat for shipping and forwarding logs to central server, etc. Similarly, we will build AndroidBeat to carry and ship device meta-data along with threat information.

ElasticBeat is developed using golang. Every beat app includes a library named libBeat which is responsible for sending operational data to Elasticsearch, either directly or via Logstash, so it can be visualized with Kibana. The challenge is to port ElasticBeats to mobile which involves cross-compiling native libraries for ARM and building the necessary scheduler for executing beats command to trigger data collection & transport.

Architecture

Traditionally, endpoints need some kind of middleware to structure data before it’s stored on backend servers for further processing. Such requirements force us to design and scale middleware APIs to handle traffic originating from endpoints. Furthermore, this setup creates further rigidity by forcing the communication between the two entities to conform to predefined data structures. Adding a single meta-data on the mobile end will need to be understood by the middleware, where to route, store, and process need to be answered.

Old Architecture

Introducing AndroidBeat

I want to short-circuit this constraint by removing components that sit between our endpoint and backend services. AndroidBeat is an ElasitcBeat app designed to collect device meta-data and forensic and send it directly to ElasticSearch. This eliminates the need to maintain a custom backend and adds the flexibility to query the collected data easily on Kibana.

New Architecture

Components

AndroidBeat - Native library built for arm/arm64/x86. It’s responsible for collecting data and shipping it to ElasticSearch Server. There is a yaml configuration file that dictates how often data should be pushed and the location of the elastic server.

Beat Configuration

Scheduler - Is responsible for scheduling how often a beat should run. Normally, on other platforms (say Linux server or container pod), the beat executable is allowed to run indefinitely under supervisord or other process control mechanisms. On Android, since resource consumption is restricted we would have to build a scheduler based on JobScheduler or AlarmManager. This not only helps reduce battery consumption but makes our app stay responsive and not get killed for misuse. This component can be implemented as a separate module and can be shared with other beat apps that need similar functionality.

Trigger - This is a component responsible for triggering the scheduler kicking off the whole process. We have two options here

  1. Via URL (Instant App) - This trigger is handy because we can simply send a URL to a user either via email or messenger to install an instant app. Instant apps are “mini” apps designed by Google that are installed directly from a URL and deliver an app-like experience.
  2. App - Installable APK containing all the components discussed above.

ELK Server - Hosted ELK server where we accumulate our data

Technicalities

There is a good guide for creating custom elastic beats here https://www.elastic.co/guide/en/beats/devguide/current/new-beat.html. The guide explains how to setup dependencies and create a basic skeleton beat application. You can follow the step by step instructions there. What’s important here is the ability to cross compile our beats app for Android/ARM. That can be achieved by using golang’s GOOS & GOARCH environment variables

GOOS=android GOARCH=arm64 go build
Use Cases

This setup greatly reduces many complexities while running an endpoint data gathering & aggregation tasks. Here are some use cases for it

  1. Vulnerability search - Search for vulnerable devices by querying device settings remotely. Hence, bringing the power of OSQuery idea to mobile
  2. Threat hunt - Search IOCs by inspecting list of installed apps & network configs to get an idea of the device’s health status.
  3. Threat intelligence for mobile
  4. Endpoint detection and response for mobile