-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathmain.py
More file actions
86 lines (74 loc) · 5.52 KB
/
Copy pathmain.py
File metadata and controls
86 lines (74 loc) · 5.52 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
import treetaggerwrapper
from read_json_data import read_json_data
from correct_word import delete_unuse_word
from correct_word import correct_word
from convert_strings import pos_to_jpos
from convert_strings import jpos_to_hinshi
from output_wordlist import export_xlsx
# Windows C:/TreeTagger/
# Mac /Users/kazuomiyajima
# コンソール入力でMac・Windows選択 #######
while True:
choice = input('please select Windows(w) or Mac(m): ')
if choice in ['w', 'Windows', 'win', 'windows']:
tagger_path = 'C:/TreeTagger/'
break
elif choice in ['m', 'mac', 'Mac']:
tagger_path = '/Users/kazuomiyajima'
break
#########################
tagger = treetaggerwrapper.TreeTagger(TAGLANG='en', TAGDIR=tagger_path) # binやlibのフォルダがあるディレクトリ
tags = tagger.TagText("Hi, Mao. I'm Felipe. Sorry, I'm not Mao. I am Mao. Oh, you are Mao. Are you from Australia ? No, I'm not. Where are you from ? I'm from Brazil. p30. I have a dog at home. Oh, nice! I have five frogs . I don't like frogs…. Do you walk your dog every day ? No, I don't . When do you walk your dog ? On weekends. p38. I can make sushi. That's nice. But I can't eat fish. Then I can make natto rolls for you. Thanks. Can you cook Japanese food ? Yes, I can . What can you make ? I can make ramen. p52. This is yokan. Oh, I like yokan. Is that yokan too ? No, it isn't. It's coffee jelly. Let's have some sweet. I like this cookie. Who is this woman ? She is a famous cook in Japan. p62. Is this your sister's work? Yes. She likes home economics. She saws very well. Thanks. But she doesn't cook. Does your sister make clothes ? Yes, she does . What can she make? She can make pajamas. p72. Do you know that cool girl? No. I don't know her . Then do you know him ? Oh, he's my brother. Do you have a girlfriend? Yes, I do. Her name is Rei. Why do you like her ? Because she is always kind to me. p82. Dad, I'm very hungry. Me too. Oh, there is a steak restaurant over there. Let's go soon! I'm full. But I want something sweet. I know a new cake shop. Wow! How can we go there ? By bus. p82. Can you help me? Sorry, I now. How about you, Kevin? OK. I'm not doing anything now. Are you still studying? No, I'm not . Then what are you doing? I'm preparing for the countdown event. p82. Tell me about your holidays. I stayed home and watched TV. You relaxed well. Yes. I also cooked nikujaga. I came back from Finland yesterday. Wow! I saw the aurora there. How lucky! Did you enjoy your trip to Finland ? Yes, I did . Do you have jet lag? Yes. I didn't sleep well last night. p82. Look. I was a student thirty years ago. Oh, my. You were very cute. You WERE? Sorry. You are still cute. How was your day? Were you busy ? No, I wasn't so busy today. Great. Let's have dinner. Awesome! What were you doing? I was reading this book. Ah, is it a picture book? Yes. I enjoyed the story.")
# タグ付きテキストを辞書型に変換
wordandtag = []
for tag in tags:
wordandtag.append({'original_word': tag.split('\t')[0].lower(), 'pos': tag.split('\t')[1], 'lemma': tag.split('\t')[2]})
# 不要な単語を削除する。句読点や数字など
wordlist = delete_unuse_word(wordandtag)
# ここで単語と品詞を修正TreeTaggerで小文字になってしまったものや、分割された省略形を正す。
wordlist = correct_word(wordlist)
# key'word'でソートをかける #lower()をつけないと大文字小文字を区別してソート(大文字が先頭に来る)。
#wordlist.sort(key=lambda x: x['word'].lower(), reverse=False)
# 配列に納められた辞書型データに一致するデータを取得
# 下の例はphoneという配列辞書型データの中にある'単語'というキーで'a.m.'にマッチするキー'発音記号'のデータを取り出す。
# next((x for x in phone if x['単語'] == 'a.m.'), None)['発音記号']でマッチした値(左の例では発音記号)
# phone['key']=''で辞書型のデータを追加できる。
# 単語の比較は原型[lemma]を使っているので注意
# 発音記号・和訳を入れるエラー処理を確実にするためループは別々にした
# 判定した品詞を判読可能な文字に変換して追加
pos_to_jpos(wordlist) # いったん日本語の品詞名に変換する
jpos_to_hinshi(wordlist) # 日本語の品詞名からHinshiフォントで表示できるアルファベットに変換する
# ここから既習後データを読み込んでデータを称号・追加
# 既習語データを読み込み
exData = read_json_data('R4ワードリスト重複削除.json')
phoneData = read_json_data('発音記号品詞H28R4.json')
# 発音記号を追加
for i in wordlist:
try:
i['phone'] = next((x for x in phoneData if x['単語'] == i['word'] and x['品詞'] == i['pos_m']), None)['発音記号']
except KeyError:
i['phone'] = ''
except TypeError:
i['phone'] = ''
# 和訳を入れる
for i in wordlist:
try:
i['mean'] = next((x for x in exData if x['単語'] == i['word'] and x['品詞'] == i['pos_m']), None)['和訳']
except KeyError:
i['mean'] = ''
'''
# 識別子(参照元)を入れる
for i in wordlist:
try:
i['identifier'] = next((x for x in exData if x['単語'] == i['lemma']), None)['識別子']
except KeyError:
i['identifier'] = ''
# 識別子に対応するページを入れる(参照元のページ)を入れる
for i in wordlist:
try:
i['page'] = next((x for x in exData if x['単語'] == i['lemma']), None)['掲載']
except KeyError:
i['page'] = ''
'''
print(wordlist)
# export_xlsx(wordlist)