Use of machine learning in big data analytics for insider threat detection

Michael Mayhew, Michael Atighetchi, Aaron Adler, Rachel Greenstadt

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    In current enterprise environments, information is becoming more readily accessible across a wide range of interconnected systems. However, trustworthiness of documents and actors is not explicitly measured, leaving actors unaware of how latest security events may have impacted the trustworthiness of the information being used and the actors involved. This leads to situations where information producers give documents to consumers they should not trust and consumers use information from non-reputable documents or producers. The concepts and technologies developed as part of the Behavior-Based Access Control (BBAC) effort strive to overcome these limitations by means of performing accurate calculations of trustworthiness of actors, e.g., behavior and usage patterns, as well as documents, e.g., provenance and workflow data dependencies. BBAC analyses a wide range of observables for mal-behavior, including network connections, HTTP requests, English text exchanges through emails or chat messages, and edit sequences to documents. The current prototype service strategically combines big data batch processing to train classifiers and real-time stream processing to classifier observed behaviors at multiple layers. To scale up to enterprise regimes, BBAC combines clustering analysis with statistical classification in a way that maintains an adjustable number of classifiers.

    Original languageEnglish (US)
    Title of host publication2015 IEEE Military Communications Conference, MILCOM 2015
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages915-922
    Number of pages8
    ISBN (Electronic)9781509000739
    DOIs
    StatePublished - Dec 14 2015
    Event34th Annual IEEE Military Communications Conference, MILCOM 2015 - Tampa, United States
    Duration: Oct 26 2015Oct 28 2015

    Publication series

    NameProceedings - IEEE Military Communications Conference MILCOM
    Volume2015-December

    Other

    Other34th Annual IEEE Military Communications Conference, MILCOM 2015
    CountryUnited States
    CityTampa
    Period10/26/1510/28/15

    Fingerprint

    Access control
    Learning systems
    Classifiers
    HTTP
    Information use
    Electronic mail
    Large scale systems
    Industry
    Processing
    Big data

    Keywords

    • big data
    • chat
    • documents
    • email
    • HTTP
    • insider threat
    • machine learning
    • support vector machine
    • TCP
    • trust
    • usage patterns

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering

    Cite this

    Mayhew, M., Atighetchi, M., Adler, A., & Greenstadt, R. (2015). Use of machine learning in big data analytics for insider threat detection. In 2015 IEEE Military Communications Conference, MILCOM 2015 (pp. 915-922). [7357562] (Proceedings - IEEE Military Communications Conference MILCOM; Vol. 2015-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MILCOM.2015.7357562

    Use of machine learning in big data analytics for insider threat detection. / Mayhew, Michael; Atighetchi, Michael; Adler, Aaron; Greenstadt, Rachel.

    2015 IEEE Military Communications Conference, MILCOM 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 915-922 7357562 (Proceedings - IEEE Military Communications Conference MILCOM; Vol. 2015-December).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Mayhew, M, Atighetchi, M, Adler, A & Greenstadt, R 2015, Use of machine learning in big data analytics for insider threat detection. in 2015 IEEE Military Communications Conference, MILCOM 2015., 7357562, Proceedings - IEEE Military Communications Conference MILCOM, vol. 2015-December, Institute of Electrical and Electronics Engineers Inc., pp. 915-922, 34th Annual IEEE Military Communications Conference, MILCOM 2015, Tampa, United States, 10/26/15. https://doi.org/10.1109/MILCOM.2015.7357562
    Mayhew M, Atighetchi M, Adler A, Greenstadt R. Use of machine learning in big data analytics for insider threat detection. In 2015 IEEE Military Communications Conference, MILCOM 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 915-922. 7357562. (Proceedings - IEEE Military Communications Conference MILCOM). https://doi.org/10.1109/MILCOM.2015.7357562
    Mayhew, Michael ; Atighetchi, Michael ; Adler, Aaron ; Greenstadt, Rachel. / Use of machine learning in big data analytics for insider threat detection. 2015 IEEE Military Communications Conference, MILCOM 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 915-922 (Proceedings - IEEE Military Communications Conference MILCOM).
    @inproceedings{63656fa081cb4ceda7365a1bbfb1aada,
    title = "Use of machine learning in big data analytics for insider threat detection",
    abstract = "In current enterprise environments, information is becoming more readily accessible across a wide range of interconnected systems. However, trustworthiness of documents and actors is not explicitly measured, leaving actors unaware of how latest security events may have impacted the trustworthiness of the information being used and the actors involved. This leads to situations where information producers give documents to consumers they should not trust and consumers use information from non-reputable documents or producers. The concepts and technologies developed as part of the Behavior-Based Access Control (BBAC) effort strive to overcome these limitations by means of performing accurate calculations of trustworthiness of actors, e.g., behavior and usage patterns, as well as documents, e.g., provenance and workflow data dependencies. BBAC analyses a wide range of observables for mal-behavior, including network connections, HTTP requests, English text exchanges through emails or chat messages, and edit sequences to documents. The current prototype service strategically combines big data batch processing to train classifiers and real-time stream processing to classifier observed behaviors at multiple layers. To scale up to enterprise regimes, BBAC combines clustering analysis with statistical classification in a way that maintains an adjustable number of classifiers.",
    keywords = "big data, chat, documents, email, HTTP, insider threat, machine learning, support vector machine, TCP, trust, usage patterns",
    author = "Michael Mayhew and Michael Atighetchi and Aaron Adler and Rachel Greenstadt",
    year = "2015",
    month = "12",
    day = "14",
    doi = "10.1109/MILCOM.2015.7357562",
    language = "English (US)",
    series = "Proceedings - IEEE Military Communications Conference MILCOM",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    pages = "915--922",
    booktitle = "2015 IEEE Military Communications Conference, MILCOM 2015",

    }

    TY - GEN

    T1 - Use of machine learning in big data analytics for insider threat detection

    AU - Mayhew, Michael

    AU - Atighetchi, Michael

    AU - Adler, Aaron

    AU - Greenstadt, Rachel

    PY - 2015/12/14

    Y1 - 2015/12/14

    N2 - In current enterprise environments, information is becoming more readily accessible across a wide range of interconnected systems. However, trustworthiness of documents and actors is not explicitly measured, leaving actors unaware of how latest security events may have impacted the trustworthiness of the information being used and the actors involved. This leads to situations where information producers give documents to consumers they should not trust and consumers use information from non-reputable documents or producers. The concepts and technologies developed as part of the Behavior-Based Access Control (BBAC) effort strive to overcome these limitations by means of performing accurate calculations of trustworthiness of actors, e.g., behavior and usage patterns, as well as documents, e.g., provenance and workflow data dependencies. BBAC analyses a wide range of observables for mal-behavior, including network connections, HTTP requests, English text exchanges through emails or chat messages, and edit sequences to documents. The current prototype service strategically combines big data batch processing to train classifiers and real-time stream processing to classifier observed behaviors at multiple layers. To scale up to enterprise regimes, BBAC combines clustering analysis with statistical classification in a way that maintains an adjustable number of classifiers.

    AB - In current enterprise environments, information is becoming more readily accessible across a wide range of interconnected systems. However, trustworthiness of documents and actors is not explicitly measured, leaving actors unaware of how latest security events may have impacted the trustworthiness of the information being used and the actors involved. This leads to situations where information producers give documents to consumers they should not trust and consumers use information from non-reputable documents or producers. The concepts and technologies developed as part of the Behavior-Based Access Control (BBAC) effort strive to overcome these limitations by means of performing accurate calculations of trustworthiness of actors, e.g., behavior and usage patterns, as well as documents, e.g., provenance and workflow data dependencies. BBAC analyses a wide range of observables for mal-behavior, including network connections, HTTP requests, English text exchanges through emails or chat messages, and edit sequences to documents. The current prototype service strategically combines big data batch processing to train classifiers and real-time stream processing to classifier observed behaviors at multiple layers. To scale up to enterprise regimes, BBAC combines clustering analysis with statistical classification in a way that maintains an adjustable number of classifiers.

    KW - big data

    KW - chat

    KW - documents

    KW - email

    KW - HTTP

    KW - insider threat

    KW - machine learning

    KW - support vector machine

    KW - TCP

    KW - trust

    KW - usage patterns

    UR - http://www.scopus.com/inward/record.url?scp=84959282598&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84959282598&partnerID=8YFLogxK

    U2 - 10.1109/MILCOM.2015.7357562

    DO - 10.1109/MILCOM.2015.7357562

    M3 - Conference contribution

    T3 - Proceedings - IEEE Military Communications Conference MILCOM

    SP - 915

    EP - 922

    BT - 2015 IEEE Military Communications Conference, MILCOM 2015

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -