--- title: "Conference Demo Spark + AI Summit 2018: State of the Art Natural Language Processing at Scale" description: "Spark’s MLlib provides a number of machine learning algorithms, and now there are also projects making deep learning achievable in MLlib pipelines. All we need is the NLP annotation frameworks. SparkNLP adds NLP annotations into the MLlib ecosystem. This talk will introduce SparkNLP: how to use it, its current functionality, and where it is going in the future." --- [![John Snow Labs](https://events.johnsnowlabs.com/hs-fs/hubfs/John_Snow_Labs_February2020/Images/johnsnowlabs_logo.png?width=1364&height=468&name=johnsnowlabs_logo.png "John Snow Labs")](https://www.johnsnowlabs.com/) - [Home](https://www.johnsnowlabs.com/) - Products - [Spark NLP](https://www.johnsnowlabs.com/spark-nlp/) - [Healthcare NLP](https://www.johnsnowlabs.com/spark-nlp-health/) - [Spark OCR](https://www.johnsnowlabs.com/spark-ocr/) - [Annotation Lab](https://www.johnsnowlabs.com/annotation-lab/) - [Live Demos](https://www.johnsnowlabs.com/demos/) - [Install Software](https://www.johnsnowlabs.com/install/) - [Install on Databricks](https://www.johnsnowlabs.com/databricks/) - [Healthcare Data Library](https://www.johnsnowlabs.com/welcome-to-the-healthcare-data-library/) - [Healthcare AI Lab](https://www.johnsnowlabs.com/healthcare-ai-platform/) - Solutions - [De-Identification](https://www.johnsnowlabs.com/deidentification/) - [Trial Master File Migration](https://www.johnsnowlabs.com/trial-master-file-migration/) - [Clinical Data Abstraction](https://www.johnsnowlabs.com/clinical-data-abstraction/) - [Clinical Decision Support](https://www.johnsnowlabs.com/clinical-decision-support/) - [Real World Data Curation](https://www.johnsnowlabs.com/real-world-data-curation/) - [Biomedical Question Answering](https://www.johnsnowlabs.com/biomedical-question-answering/) - [Customers](https://www.johnsnowlabs.com/customers/) - [Company](https://www.johnsnowlabs.com/our-story) - [Our story](https://www.johnsnowlabs.com/our-story) - [Our Team](https://www.johnsnowlabs.com/our-team) - [Careers](https://www.johnsnowlabs.com/careers/) - [Awards & Press](https://www.johnsnowlabs.com/awards/) - [Social Impact](https://www.johnsnowlabs.com/social-impact/) - [Announcements](https://www.johnsnowlabs.com/announcements/) - [Learn](https://www.johnsnowlabs.com/resources/) - [NLP Summit](https://www.nlpsummit.org/) - [Webinars](https://events.johnsnowlabs.com/webinars) - [Training](https://www.johnsnowlabs.com/spark-nlp-training/) - [Peer-Reviewed Papers](https://www.johnsnowlabs.com/peer-reviewed-papers/) - [NLP for business Leaders](https://www.johnsnowlabs.com/nlp-for-business-leaders/) - [AI in Healthcare Blog](https://www.johnsnowlabs.com/ai-and-healthcare-blog/) - [Spark NLP Blog](https://www.johnsnowlabs.com/spark-nlp-blog/) - [Spark NLP for Academia](https://www.johnsnowlabs.com/spark-nlp-for-research-and-education/) - [Install Software for Free](https://www.johnsnowlabs.com/install/) - [Schedule a Call](https://www.johnsnowlabs.com/schedule-a-demo/) - - [Home](https://www.johnsnowlabs.com/) - [Spark NLP](https://www.johnsnowlabs.com/spark-nlp) - [State of the Art NLP](https://www.johnsnowlabs.com/spark-nlp) - [NLP For Healthcare](https://www.johnsnowlabs.com/spark-nlp-health/) - [Spark OCR](https://www.johnsnowlabs.com/spark-ocr/) - [Live Demos](https://www.johnsnowlabs.com/spark-nlp-in-action/) - [Free Trial](https://www.johnsnowlabs.com/spark-nlp-try-free/) - [Case Studies](https://www.johnsnowlabs.com/nlp-case-studies/) - [Healthcare AI](https://www.johnsnowlabs.com/ai-platform) - [De-Identification](https://www.johnsnowlabs.com/deidentification/) - [Annotation Lab](https://www.johnsnowlabs.com/annotation-lab/) - [Healthcare AI Lab](https://www.johnsnowlabs.com/ai-platform/) - [AI Lab Architecture](https://www.johnsnowlabs.com/healthcare-ai-platform-features/) - [Data Library](https://www.johnsnowlabs.com/welcome-to-the-healthcare-data-library/) - [AI Case Studies](https://www.johnsnowlabs.com/ai-case-studies/) - [Company](https://www.johnsnowlabs.com/our-story) - [Our story](https://www.johnsnowlabs.com/our-story) - [Our Team](https://www.johnsnowlabs.com/our-team) - [Careers](https://www.johnsnowlabs.com/careers/) - [Awards & Press](https://www.johnsnowlabs.com/awards/) - [Social Impact](https://www.johnsnowlabs.com/social-impact/) - [Announcements](https://www.johnsnowlabs.com/announcements/) - [Learn](https://www.johnsnowlabs.com/resources/) - [NLP Summit](https://www.nlpsummit.org/) - [Webinars](https://events.johnsnowlabs.com/webinars) - [Training](https://www.johnsnowlabs.com/spark-nlp-training/) - [Papers & Conferences](https://events.johnsnowlabs.com/conferences-videos?_ga=2.12548788.1674323299.1603701754-1154577709.1587485642) - [AI in Healthcare Blog](https://www.johnsnowlabs.com/ai-and-healthcare-blog/) - [Spark NLP Blog](https://www.johnsnowlabs.com/spark-nlp-blog/) - [Spark NLP for Academia](https://www.johnsnowlabs.com/spark-nlp-for-research-and-education/) - [Schedule a Demo](https://www.johnsnowlabs.com/schedule-a-demo/) - [Free Trial](https://www.johnsnowlabs.com/spark-nlp-try-free/) - [Account](https://www.johnsnowlabs.com/account/) #### Spark + AI Summit 2018 #### State of the Art Natural Language Processing at Scale   Natural language processing is a key component in many data science systems that must understand or reason about text. Common use cases include question answering, paraphrasing or summarization, sentiment analysis, natural language BI, language modeling, and disambiguation. Building such systems usually requires combining three types of software libraries: NLP annotation frameworks, machine learning frameworks, and deep learning frameworks. Ideally, all three of these pieces should be able to be integrated into a single workflow. This makes development, experimentation, and deploying results much easier. Spark’s MLlib provides a number of machine learning algorithms, and now there are also projects making deep learning achievable in MLlib pipelines. All we need is the NLP annotation frameworks. SparkNLP adds NLP annotations into the MLlib ecosystem. This talk will introduce SparkNLP: how to use it, its current functionality, and where it is going in the future. ###### About the speaker ![51127-speaker](https://events.johnsnowlabs.com/hs-fs/hubfs/51127-speaker.jpeg?width=150&name=51127-speaker.jpeg) David Talby is a CTO at John Snow Labs, helping fast-growing companies apply big data and data science techniques to solve real-world problems in healthcare, life science, and related fields. David has extensive experience in building and operating web-scale data science and business platforms, as well as building world-class, Agile, distributed teams. Previously, he was with Microsoft’s Bing Group, where he led business operations for Bing Shopping in the US and Europe and worked at Amazon both in Seattle and the UK, where he built and ran distributed teams that helped scale Amazon’s financial systems. David holds a Ph.D. in computer science and master’s degrees in both computer science and business administration. ### Recent blog posts ### Get to work Need NLP Software? [Try Free](https://www.johnsnowlabs.com/spark-nlp-try-free/) Need NLP in Healthcare? [Schedule a Demo](https://www.johnsnowlabs.com/schedule-a-demo/) Need to Learn More? [Register for NLP Summit](https://www.nlpsummit.org/) #### Let’s Talk! 16192 Coastal Highway Lewes, DE 19958, USA [info@johnsnowlabs.com](mailto:info@johnsnowlabs.com) [+1 (302) 786-5227](tel:+13027865227) ### Join our newsletter © 2020 John Snow Labs. John Snow Labs Inc. [Terms of Service](https://www.johnsnowlabs.com/terms-of-service/) | [Refund Policy](https://www.johnsnowlabs.com/refund-policy/) | [Privacy Policy](https://www.johnsnowlabs.com/privacy-policy)