{"id":1029,"date":"2017-07-27T11:44:00","date_gmt":"2017-07-27T09:44:00","guid":{"rendered":"https:\/\/ilc4clarin.ilc.cnr.it\/en\/?p=1029"},"modified":"2024-03-20T20:11:56","modified_gmt":"2024-03-20T19:11:56","slug":"multiword-extractor","status":"publish","type":"post","link":"https:\/\/ilc4clarin.ilc.cnr.it\/en\/multiword-extractor\/","title":{"rendered":"Multiword Extractor"},"content":{"rendered":"

\"en_GB\"\/<\/a>\"it_IT\"\/<\/a><\/p>\n

Language-independent multiword extractor.<\/p>\n

This is a PANACEA service. It extracts all possible candidate multiwords from POS tagged text in conll format starting from a pair of POS (of the first and last words of the pattern) in a given window size. The user must know the tagset used in the data in order to properly set the parameters.<\/p>\n

Input<\/strong>: a conll-07 POS tagged text file (dependency analysis is not required, but dependency annotated text are accepted)<\/p>\n

Output options<\/strong>:
\nTSV: tabular text format
\nXML: LMF-XML lexicon data<\/p>\n

Optional parameters<\/strong>:
\napos<\/em> = POS of the first word of the search space
\nbpos<\/em> = POS of the last word of the search space
\ndomain<\/em> = Label for the thematic or technical domain of the corpus (for instance: LABOUR, NEWS etc.)
\nfiltering_type<\/em> = type of filtering for the full multiword (First, Overmean, Sigma)
\nmax_entry_num<\/em> = the number of total candidate to be shown in the results\/ inserted in the output lexicon (by default the service prints all possible candidate multiwords that pass the filter thresholds)
\norder_by<\/em>: set the orderin which the candidates ar displayed according to raw frequency (frequency), relative frequency (frelative), loglikelyhood (ll), pointwise mutual information (mi)
\noutput_type<\/em>: tsv or lmf
\nprefiltering_type<\/em>: this is a filter based on the statistics on the word pairs, i.e before the actual full MW expressions are extracted. Possible options: average frequency (averagef), maximum frequency (maxf)
\nproperty_file<\/em>: the user may set all these parameters in a single text file to be passed to the service
\nwindow<\/em>: a digit indicating the size of the window for the search space; i.e. the maximum size in terms of words for the candidate expressions to be extracted (for instance: 3)<\/p>\n

Please note: the service potentially works on windows of size n; however, it has been tested with a max value of 5.<\/p>\n

The tool functionalities and filtering methods are detailed here<\/a>.
\nThe code is available here<\/a>.<\/p>\n

URL: SCF Extractor (lang indip)<\/a> (WSDL<\/a>)<\/p>\n

Loading… please wait.<\/h3>\n