The course project amounts to 40% of the course grade. The course project requires students to (1) build an Android application adhering to security best-practices, as well as to (2) analyze a corpus of Android applications for finding vulnerabilities. By completing the project, students will get a first-hand experience of the challenges of designing secure mobile applications, as well as the pitfalls and errors made by developers in the process.
The course project will be composed of two phases, each amounting to 20% of the course grade (i.e., the credit for the course project will be evenly divided among the two phases). The first phase, “Application Design”, is expected to be completed (and hopefully graded) around Fall break, whereas the second phase (“Application Analysis”) begins after Fall break, and lasts until the last day of classes. In addition, up to 5 extra credit points on the course grade may be awarded for the project presentation at the end of the semester.
Project teams may include groups of up to four students; however, groups of greater size will be expected to make greater progress. I will advise each team/individual independently as needed. The project grade will be a combination of grades received for a number of milestone artifacts and the final project report.
See the class Schedule for milestone due dates.
NOTE: Although many of the concepts learned in this class apply to iOS applications as well, we will only be developing and analyzing Android applications in this course project.
The course project is out of 200 points distributed as follows:
- Milestone 1: Project proposal 10 points (Phase 1: Secure Mobile Application Design)
- Milestone 2: Application Design and Implementation 90 points (Phase 1: Secure Mobile Application Design)
- Milestone 3: Analysis Plan 40 points (Phase 1: Mobile Application Security Analysis)
- Milestone 4: Analysis Report 60 points (Phase 1: Mobile Application Security Analysis)
- Milestone 5: Project Presentation (with questions) 5 Extra Credit on course grade
Milestone 1: Project proposal (10 points)
The purpose of this milestone is to settle on 1) an application idea, and 2) a project team. The mobile application idea and team must be approved by the instructor.
Each team will upload the project proposal containing 5+ unique project ideas before the specified deadline, and then meet with the instructor. Your grade on this milestone will depend on the team’s ability to decide on at least one good project idea during this meeting with the instructor. I highly recommend having 5+ unique project ideas (not slight variations) for this meeting.
Each proposed mobile application should possess at least 4 characteristics of the ones listed below:
- Network Communication: Send and receive data over the network
- Storage: Store data on the device; especially, secrets (e.g., passwords).
- Permissions: Access permission-protected API on the device.
- Inter-application communication: Provide a service or an interface to other apps.
- Webviews: Use WebViews to load Web pages within the app.
- Sensors/ Sensitive API: Use device sensors/camera/microphone/ or some other sensitive API
Milestone 2: Application Design and Implementation (90 points)
Students will design and implement the Android application decided at the end of the project proposal meeting as a part of this milestone. For grading this milestone, the following criteria will be considered:
- Functionality: The application should work as described. Points will be deducted for every feature that crashes. Simple, well-tested applications will successfully pass this criteria.
- Security: Adherence to the security best-practices taught in class is key to scoring well in this milestone. Points will be heavily deducted for exhibiting common security pitfalls taught in class.
NOTE: No points will be awarded on the milestone for apps that don’t compile, or crash on start.
What to turn in:Each team will submit three artifacts, in a single compressed archive (lastname.tar.gz or lastname.zip):
Project document: A single-page document (lastname.pdf) that clearly describes the application, its features, and how the application should be used. The description must be in layman terms (e.g., similar to descriptions on Application markets), and must be understood by the TA for the milestone to be graded.
Application source code: A compressed archive (lastname-source.tar.gz or appname-source.zip) containing the entire source code of the application, including any images/libraries necessary to compile the application from scratch. The TA should be able to build the application solely using this source code archive, without any additional libraries.
SIGNED APK: An Android application package (i.e., lastname.apk) that is signed with the team’s certificate (and not the default certificate packed with the IDE).
Project Roles & Responsibilities Document: A description of the role and precise contribution of each group member in the project (i.e., lastname-contributions.pdf).
Note about files: Only .tar.gz, and .zip files will be accepted (internal files may be .apk, .pdf). Filenames must follow the lastname- convention. If more than one student is on a project, the file name should have the prefix lastname1-lastname2-lastname3-lastname4, where the lastnames are in alphabetical order. Failure to comply with any of these file format and naming requirements will result in an automatic five point deduction from the milestone grade.
Milestone 3: Analysis Plan (40 points)
In the second phase of the project (i.e., milestones 3 and 4), students will analyze a set of Android applications for security vulnerabilities and/or malicious behavior. The purpose of this milestone is to set the scope, and outline the research methods/techniques to be used in the actual analysis.
You will analyze a set of 50-100 apps, for 3 unwanted/vulnerable behaviors:
- Permission misuse: An application’s use of permissions may violate the user’s privacy/security. Understanding Android permissions, and what combinations of permissions may be risky, is critical.
- SSL API misuse: An application’s network communication may be vulnerable (i.e., due to not using SSL, or using it incorrectly)
- Interface Vulnerabilities: An application may have vulnerable interfaces, or may not sanitize data received from other applications.
You must describe the following in an analysis plan (at least 2 full pages), modifying the given analysis plan template:
Research Questions (20 points): A list of at least 3 precise (more desired) questions for each analysis goal (i.e., at least 9 in total) that your analysis will potentially answer, for the given set of apps. For example, “Do applications take input from other apps, and use it without sanitizing it?”, or “Do apps use HTTPS for network communication? or, “Do apps verify certificates incorrectly?”, or “Do apps use combinations of permissions that may potentially be used in a harmful manner?”, or “Do apps request more permissions than they use”, are all valid questions. Research questions may also be specific to the analysis; e.g., is the proposed analysis practical (i.e., performance in terms of runtime and accuracy)?
Evaluation Plan (20 points): A description of how you plan to answer your research questions. The evaluation plan will describe a set of experiments. The experiments may mirror the RQs. Go into as much detail as possible. The plan should ideally describe (1) what the experiment tests, (2) the specific technique(s) (e.g., simple string searches on the AndroidManifest, analyzing specific classes or methods, manual analysis or confirmation), and (3) success criteria (i.e., what constitutes success, and what constitutes as a false positive).
What to turn in: PDF of the analysis plan (lastname-plan.pdf).
Note about files: Only .pdf. Filenames must follow the lastname- convention. If more than one student is on a project, the file name should have the prefix lastname1-lastname2-, where the lastnames are in alphabetical order. The plan must be written using LaTeX and citations must be managed in one or more .bib files using BibTeX. Failure to comply with any of these file format and naming requirements will result in an automatic five point deduction from the milestone grade.
Milestone 4: Analysis Report (60 points)
For the analysis, each group will write scripts, perform light-weight static analysis, or manual analysis, to answer their respective research questions. Dynamic analysis may be performed, but given the limited time and resources at hand, is not recommended.
Team-based requirements Since the teams range from sizes 1–>4, the required effort will be proportionate to the team size. Here is the minimum effort required, based on the team size:
- Team of 1: At least one RQ from each of the three research goal (i.e., 3 RQs total).
- Team of 2: At least 5 RQs, with at least one RQ from each of the three research goals.
- Team of 3: At least 7 RQs.
- Team of 4: At least 9 RQs.
Project Presentations Up to 5 bonus points will be awarded for a 15 minute project presentation. The presentations will describe the research plan, current progress, current results, and anticipated results and experiments. That is, you are not expected to have completed your project at this point. Each team is encouraged to present.
Important Note on using past research tools: You will not be allowed to use research/commercial tools developed for answering your specific research questions; however, you can use and build upon general-purpose tools. For example, consider research questions related to SSL verification flaws. You cannot use the MalloDroid tool directly; however, you can use the lessons/research questions from the MalloDroid paper, or write Androguard scripts the way MalloDroid did. If you have questions, please contact the instructor.
The application analysis phase will culminate in a written report consisting of minimum 5 pages, not including references, 1-inch margins, two column, 10-pt font. Modify the final report template for this milestone.
The grading will be based on the depth of the analysis, the findings, effort, as well as the quality of the writing. Here is a suggested outline:
- Evaluation outline/overview
- Experiments (detailed description)
- Findings/Interpretation of Results
- Takeaway and Conclusions
What to turn in: A .tar.gz or .zip containing (1) the PDF of the final report (lastname-report.pdf) and (2) the source code of the analyses.
Note about files: Filenames must follow the lastname- convention. If more than one student is on a project, the file name should have the prefix lastname1-lastname2-, where the lastnames are in alphabetical order. The report must be written using LaTeX and citations must be managed in one or more .bib files using BibTeX. Failure to comply with any of thee file format and naming requirements will result in an automatic five point deduction from the milestone grade.