Posted on 11/05/2024 12:00:00
Why not? Digital preservation is important, even if you don’t care for a specific program. This is also a good way to get started with protocol reverse engineering due to way protobuf is often left behind in source format within client applications.
In this series of blogposts, I will be using a mobile game “Egg, Inc.” as the target for demonstration. It’s a simple time killer app that got me through boring long waits when I was still at school.
Egg, Inc. is a basic incremental “idler” game where your goal is to take over the world food supply with ever-increasing supply of eggs, if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and “Eggs of Prophecy” which increase potency of your Soul Eggs.
It’s rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that’s beside the point. The simplicity of our target matters here.
In some cases, you will find previous works on the target you pick. In my case, some clever people have created scripts to extract .proto file out of the app. I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the API spec .proto file for your target.
Further there are a few dedicated individuals in the game’s community who have created numerous tools and historical databases.
For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server.
There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver
and extract it from /data/app/ - identifying the app by its icon. From there you will find base.apk which is enough for most apps.
Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select “Manual Download” and enter a known Build ID.
Thanks to the previously mentioned script, it’s easy to get started - find the APK, extract protobuf spec file, convert it with protoc and we’re done there. One small problem - due to cheaters, latest version of the game includes “AuthenticatedMessage” structure, which contains a salted sha256sum of the payload message.
At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information is retained in the app itself.
Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store).
With all that out of the way, lets get into actual commands used here:
git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git
cd EggIncProtoExtractor
./apkextract.sh com.auxbrain.egginc_1.12.13.apk
# We should have a new folder "protos" now with resulting files
cd protos
# There should be a file called ei.proto - that's our protobuf spec file
# At this point, we can use the protoc utility which can convert the specfile
# to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with
# additional plugin support for Dart and Go.
# To make this easier to understand, we will use Python in this demonstration
protoc -I=. --python_out=. ./ei.proto
# Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs
With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there’s a slight problem. What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be using Ghidra myself.
(Note: You can also just try to find this out using tools such as WireShark)
The game contains a linked-library written in C++, which you can find inside the .apk lib folder, named as libegginc.so.
This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice
and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute.
Once that’s done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left
behind or maybe some clues. I started by searching for http, which lead me to following string "HTTP REQ: %d", seems promising.
When I jumped to it, I saw an exactly adjacent string to it which could give more clues:
s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R)
00c02b5e 47 3f 77 ds "G?www.auxbrain.com"
77 77 2e
61 75 78
s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*)
00c02b71 48 54 54 ds "HTTP REQ: %d"
50 20 52
45 51 3a
Interesting, www.auxbrain.com. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up
certain global values.
So we have a potential API endpoint, let’s put it to the test. We can do a quick smoke test by setting up a webserver.
Install AdAway app from F-Droid so we can setup a redirection on any network we are on. Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server.
(NOTE: AdAway doesn’t detect any subdomains nor can it do wildcard, you will need to include the FQDN of the API endpoint www.auxbrain.com)
Once you’re done setting up the redirection, run any webserver such as nginx for a quick and dirty test.
192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-"
Bingo. We have contact and we have an API endpoint. Searching for “ei/” in the strings reveals a extensive list of API endpoints, we now have something to go off from. We have everything we need to start creating the server.
Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly. If you are following along, get respective packages for your operating system to create python venvs. As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx.
# Lets stage the environment
mkdir apiserver
cd apiserver
python -m venv .venv
source .venv/bin/activate
touch app.py
cp ~/EggIncProtoExtractor/protos/ei.proto .
# Get some dependeices
pip install protobuf
pip install flask
We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it. Let’s create an actual listener application, open app.py with your favourite IDE or text editor.
import ei_pb2 as EIProto
from flask import Flask
from flask import request
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
print(request.headers)
return ""
This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains. At this point you should setup the reverse proxy, override your nginx / directive with:
location / {
proxy_pass http://127.0.0.1:5000;
}
Reload your nginx and start the flask application you just created with flask run.
Run the app again and have it phone home and see what it contains.
HTTP POST /ei/first_contact
Host: 127.0.0.1:5000
Connection: close
Content-Length: 37
Content-Type: application/x-www-form-urlencoded
User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001)
Accept-Encoding: gzip
We can see there’s a form payload attached to this request, let’s modify our app route a bit:
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
print(request.form)
return ""
Now if we run the modified flask application again, we see following output on the first_contact endpoint.
HTTP POST /ei/first_contact
ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')])
We have a base64-encoded protobuf binary data - which isn’t terribly useful for reading plain-text, since protobuf is a binary format, so we will need to figure out what protobuf message this payload belongs to.
Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit. We know we just contacted something called “first_contact”, maybe there is something in that file that could help us?
message EggIncFirstContactRequest {
optional string user_id = 1;
optional uint32 client_version = 2;
optional Platform platform = 3;
}
message EggIncFirstContactResponse {
optional Backup backup = 1;
}
Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove to be useful knowledge. We now know what the payload should be, lets put this to the test.
Edit your app routine again
# add "import base64" to top of the file
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
if subpath == "first_contact":
# Create the protobuf object so we can load data from the b64 payload
FirstContact = EIProto.EggIncFirstContactRequest()
FirstContact.ParseFromString(base64.b64decode(form["data"]))
print(FirstContact)
else:
print(request.form)
return ""
We should now be able to see deserialized output when we run the flask application and the mobile app, let’s try it out:
HTTP POST /ei/first_contact
user_id: "355e46e9089d1e4c"
client_version: 0
platform: DROID
Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess on what would come next.
Seeing how we got EggIncFirstContactRequest and saw an adjacent EggIncFirstContactResponse message in the proto file, we
can safely assume that this is what the game is expecting from us in return.
Lets modify the server a bit to account for that.
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
if subpath == "first_contact":
# Create the protobuf object so we can load data from the b64 payload
FirstContact = EIProto.EggIncFirstContactRequest()
FirstContact.ParseFromString(base64.b64decode(form["data"]))
print("We got a first contact hello from user " + FirstContact.user_id)
# Lets respond with a FirstContactResponse
FirstContactResp = EIProto.EggIncFirstContactResponse()
# This takes only one optional argument - a Backup object - but we have no account
# system yet, so we will opt out of sending that for now.
# ---
# We send the payload back as a base64 string - the same way we retrieved it.
return base64.b64encode(FirstContactResp.SerializeToString())
else:
print(request.form)
return ""
Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends
Say hello to /ei/save_backup and /ei/get_periodicals. We can infer from the name, that save_backup would involve a Backup message
and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file.
Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again.
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
if subpath == "first_contact":
# Create the protobuf object so we can load data from the b64 payload
FirstContact = EIProto.EggIncFirstContactRequest()
FirstContact.ParseFromString(base64.b64decode(form["data"]))
print("We got a first contact hello from user " + FirstContact.user_id)
# Lets respond with a FirstContactResponse
FirstContactResp = EIProto.EggIncFirstContactResponse()
# This takes only one optional argument - a Backup object - but we have no account
# system yet, so we will opt out of sending that for now.
# ---
# We send the payload back as a base64 string - the same way we retrieved it.
return base64.b64encode(FirstContactResp.SerializeToString())
elif subpath == "save_backup":
# NOTE: This took me way longer to realize than it should have, but the base64
# payload you receive from client is broken due to some Android bug, where it
# substitutes "+" symbols with a " " whitespace.
# I don't want you to waste half hour to hours figuring out why you're getting
# corrupted data, so you're welcome.
Backup = EIProto.Backup()
Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
print(Backup)
elif subpath == "get_periodicals":
Periodicals = EIProto.GetPeriodicalsRequest()
Periodicals.ParseFromString(base64.b64decode(form["data"])
print(Periodicals)
else:
print(request.form)
return ""
We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather try this out yourself - the Backup structure is VERY large).
Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively thin payload of PeriodicalsRequest comes through, which isn’t very useful by itself.
Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place to ensure you’re not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture.
The game has an In App Purchase for “Pro Permit”, which allows you to build more Silos, which in turn allow you to get offline rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called “permit_level”, which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game.
cache = {}
@app.route("/ei/<path:subpath>", methods=["POST"])
def ei_routes(subpath):
print("HTTP POST /ei/" + subpath)
if subpath == "first_contact":
# Create the protobuf object so we can load data from the b64 payload
FirstContact = EIProto.EggIncFirstContactRequest()
FirstContact.ParseFromString(base64.b64decode(form["data"]))
print("We got a first contact hello from user " + FirstContact.user_id)
# Lets respond with a FirstContactResponse
FirstContactResp = EIProto.EggIncFirstContactResponse()
if FirstContact.user_id in cache:
FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id])
del cache[FirstContact.user_id]
return base64.b64encode(FirstContactResp.SerializeToString())
elif subpath == "save_backup":
# NOTE: This took me way longer to realize than it should have, but the base64
# payload you receive from client is broken due to some Android bug, where it
# substitutes "+" symbols with a " " whitespace.
# I don't want you to waste half hour to hours figuring out why you're getting
# corrupted data, so you're welcome.
Backup = EIProto.Backup()
Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
if Backup.game.permit_level == 0:
print("Saved a modified Backup for next game load")
# Modify the permit level, force offer the backup
Backup.game.permit_level = 1
Backup.force_offer_backup = True
Backup.force_backup = True
cache[Backup.user_id] = Backup
elif subpath == "get_periodicals":
Periodicals = EIProto.GetPeriodicalsRequest()
Periodicals.ParseFromString(base64.b64decode(form["data"])
print(Periodicals)
else:
print(request.form)
return ""
Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the “Saved a modified Backup for next game load” message show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a Backup from server. Let’s accept that.
Now click on your silos, you have the Pro Permit for free.
Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so. If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!).
This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next.
This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into the game disassembly again.
By public knowledge, we know there are server events for “Epic Research Sale”, “Research Sale”, “Drone Bonus” and “Prestige Boost”. We can use this information to try and look at any potential leads in strings. Drone seems like a good canditate, lets look into that.
drone_fans2
drone_crash
drone_enemy
drone_hunter
r_icon_drone_rewards
b_icon_drone_boost
drone_touch
ei_drone_lights_green
ei_drone_lights_red
ei_drone_package
ei_drone_propeller
drone-boost
GENEROUS DRONES
This looks promising, right off the bat, first strings I’d check here are r_icon_drone_rewards, b_icon_drone_boost, drone-boost and GENEROUS DRONES.
I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13
piggy-boost (Rate piggy fills is increased.)
piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.)
prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.)
earnings-boost (CASH BOOST;Regular earnings are increased.)
gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.)
drone-boost (GENEROUS DRONES;Drones will produce larger rewards.)
epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.)
vehicle-sale (VEHICLE SALE;Applies to all vehicles.)
boost-sale (BOOST SALE;Applies to the gold price of boosts.)
boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.)
I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event.
We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events.
elif subpath == "get_periodicals":
# We don't actually need the information client sends us,
# we aren't verifying any stats about client in our server.
CurrentPeriodicals = EIProto.PeriodicalsResponse()
# In order to add items to a repeatable field in protobuf structure,
# we need to call .add() method on it
event = CurrentPeriodicals.events.events.add()
# Refer to ei.proto - we are filling fields for EggIncEvent structure here.
event.type = "drone-boost"
event.multiplier = 5.00
event.subtitle = "Drones will produce larger rewards."
event.identifier = "GENEROUS DRONES"
event.seconds_remaining = 300.0
# Lets make it respond with a 5 minute event (this will re-arm itself when client calls
# for get_periodicals again every 6 minutes)
return base64.b64encode(CurrentPeriodicals.SerializeToString())
Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game.
We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of trusting client in server. We also created a very basic server event, which always rearms itself to never expire.
What do we do next?
At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress.
As we progress the game and start performing prestiges, we unlock a feature called “Contracts” - but disaster strikes as we don’t have any contracts we could
accept. So far we still see our good friends /ei/get_periodicals and /ei/save_backup hammering the server at regular intervals.
When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called ContractsResponse contracts. Lets see
what this ContractsResponse message contains.
message ContractsResponse {
repeated Contract contracts = 1;
optional string warning_message = 4;
optional double server_time = 2;
optional uint32 max_eop = 3 [default = 1000];
}
Notice there being an array of Contract messages right off the bat - lets find its message structure next:
message Contract {
optional string identifier = 1;
optional string name = 9;
optional string description = 10;
optional Egg egg = 2;
repeated Goal goals = 3;
message Goal {
optional GoalType type = 1;
optional double target_amount = 2;
optional RewardType reward_type = 3;
optional string reward_sub_type = 4;
optional double reward_amount = 5;
optional double target_soul_eggs = 6;
}
repeated GoalSet goal_sets = 16;
message GoalSet {
repeated Goal goals = 1;
}
optional bool coop_allowed = 4;
optional uint32 max_coop_size = 5;
optional uint32 max_boosts = 12;
optional double minutes_per_token = 15 [default = 60];
optional double expiration_time = 6;
optional double length_seconds = 7;
optional double max_soul_eggs = 13;
optional uint32 min_client_version = 14;
optional bool debug = 11;
}
We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an older revision of Contracts wiki page from 2021 and did some slight research.
From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where beginners will get easier contract goals while more advanced players get harder contract goals.
We can put two-and-two together here and infer that repeated Goal goals is the legacy contract system - where everyone was on equal footing
and repeated GoalSet goal_sets is the new goal system that is split into Standard and Elite.
We also learn that in future game versions, they completely reworked how contracts work yet again into a grading “bracket” system. Fortunately, we do not have to worry about that in our current target revision.
Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into contracts by creating a simple & easy contract called Your First Contract.
This page tells us all the information we need to compose our first Contract, so lets try to make one.
elif subpath == "get_periodicals":
# We don't actually need the information client sends us,
# we aren't verifying any stats about client in our server.
CurrentPeriodicals = EIProto.PeriodicalsResponse()
# [...]
Contract = CurrentPeriodicals.contracts.contracts.add()
Contract.identifier = "first-contract"
Contract.name = "Your First Contract"
Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!"
Contract.egg = EIProto.Egg.MEDICAL
Contract.coop_allowed = False
Contract.minutes_per_token = 5
# Lets set expiry time to always be 3 days into future
Contract.expiration_time = time.time() + (3600.0 * 72.0)
Contract.length_seconds = 3600.0 * 4.0
# The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs
Contract.max_soul_eggs = 5000.0
# We should have the basic metadata set now, lets create the goalsets.
FirstSet = Contract.goal_sets.add()
Goal = FirstSet.goals.add()
# There is only one type of goal in this verison
Goal.type = EIProto.GoalType.EGGS_LAID
Goal.target_amount = 100000.0
Goal.reward_type = EIProto.RewardType.GOLD
Goal.reward_amount = 192
Goal = FirstSet.goals.add()
Goal.type = EIProto.GoalType.EGGS_LAID
Goal.target_amount = 500000000.0
Goal.reward_type = EIProto.RewardType.PIGGY_FILL
Goal.reward_amount = 10000
# Lets now add the Elite table, we can pretty much copy-paste the above here.
SecondSet = Contract.goal_sets.add()
Goal = SecondSet.goals.add()
Goal.type = EIProto.GoalType.EGGS_LAID
Goal.target_amount = 100000.0
Goal.reward_type = EIProto.RewardType.GOLD
Goal.reward_amount = 500
Goal = SecondSet.goals.add()
Goal.type = EIProto.GoalType.EGGS_LAID
Goal.target_amount = 500000000.0
Goal.reward_type = EIProto.RewardType.PIGGY_FILL
Goal.reward_amount = 10000
return base64.b64encode(CurrentPeriodicals.SerializeToString())
Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. The contract goals are swapped! I am getting Elite contract rewards for a Standard contract.
This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. After swapping the sets around, we now see a contract with the corrected rewards.
I playtested it a bit and the contract worked as expected.
Now, the above code could be a lot neater. For your homework, if you’re not skipping to the public source release in the end, you should try to create a contract database and try scheduling them like the game originally did - a “Leggacy” contract every Friday and regular contracts showing up every 1-2 weeks for roughly 2 weeks.
We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready. If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back.
Before I give you the public source to the project, you might want to try your hand at creating a few more things.
I apologize if my method of documenting this has been messy, but that’s also part of the chaos of reverse engineering, you are constantly learning new things about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree.
I won’t give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links: github.com, git.based.quest.
Next time we will dive into apps that use SSL/TLS and making onboarding for your friends easier.
Thank you for reading and making it all the way to the end,
Join us at Matrix: #based-quest:cernodile.com
© 2021 - 2024 based.quest | Powered by Hugo | Donate | Atom RSS