Skip tokenisation and detokenisation #7
Description
CTranslate does not provide an API that takes numerical token Ids, thus we convert the IDs we get from Triton to subwords for the encoder input and convert the decoder outputs back to IDs. This is unnecessary since it means that it has to be done on both the client and server.