Skip to content

Commit 2154825

Browse files
Merge pull request #145 from artefactory/feature/readme
Feature/readme
2 parents c0acd7b + dc70a2d commit 2154825

2 files changed

Lines changed: 19 additions & 3 deletions

File tree

README.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
NLPretext
22
==============================
33

4-
### **No more pretext for dirty text** :pencil:
5-
4+
<p align="center">
5+
<img src="/references/logo_nlpretext.png" />
6+
</p>
67

78
> *Working on an NLP project and tired of always looking for the same silly preprocessing functions on the web?* :tired_face:
89
@@ -21,6 +22,7 @@ NLPretext packages in a **unique** library all the text **preprocessing** functi
2122
* [Replacing phone numbers](#replace_phone_numbers)
2223
* [Removing hashtags](#remove_hashtags)
2324
* [Extracting emojis](#extract_emojis)
25+
* [Data augmentation](#data_augmentation)
2426

2527

2628
Cannot find what you were looking for? Feel free to open an [issue]((https://github.com/artefactory/nlpretext/issues) ).
@@ -34,7 +36,7 @@ This package has been tested on Python **3.6**, **3.7** and **3.8**.
3436
To install this library you just have to run the following command:
3537

3638
```bash
37-
pip install git+https://github.com/artefactory/NLPretext.git
39+
pip install nlpretext
3840
```
3941

4042
We strongly advise you to do the remaining steps in a virtual environnement.
@@ -133,6 +135,20 @@ print(example)
133135
# [':grinning_face:']
134136
```
135137

138+
# Data augmentation <a name="data_augmentation"></a>
139+
140+
The augmentation module helps you to **generate new texts** based on your given examples by modifying some words in the initial ones and to **keep associated entities unchanged**, if any, in the case of **NER tasks**. If you want words other than entities to remain unchanged, you can specify it within the `stopwords` argument. Modifications depend on the chosen method, the ones currently supported by the module are **substitutions with synonyms** using Wordnet or BERT from the [`nlpaug`](https://github.com/makcedward/nlpaug) library.
141+
142+
```python
143+
from nlpretext.augmentation.text_augmentation import augment_text
144+
example = "I want to buy a small black handbag please."
145+
entities = [{'entity': 'Color', 'word': 'black', 'startCharIndex': 22, 'endCharIndex': 27}]
146+
example = augment_text(example, method=”wordnet_synonym”, entities=entities)
147+
print(example)
148+
# "I need to buy a small black pocketbook please."
149+
```
150+
151+
136152
# Make HTML documentation
137153

138154

references/logo_nlpretext.png

191 KB
Loading

0 commit comments

Comments
 (0)