https://github.com/yandex/faster-rnnlm

 
 

Gdb ./rnnlm

r -rnnlm model-good.faster -train thread.title.good.train.txt -valid thread.title.good.valid.txt -hidden 5- -direct-order 3 -direct 200 -bptt 4 -bptt-block 10 -threads 1

[root@cq01-forum-rstree01.cq01.baidu.com faster-rnnlm]# more thread.title.good.train.txt

唉        稳        凉菜        干货        批发        稳        左成        个        月        都

咦        丢        图        跑

毕竟        新人

我        想去旅行

昨天        玩        个        满        深渊        人马                 才        踩        了        55

这        状态        还        不如        温网

新型        投资项目

晒        早饭        就        酱

渣土        哥        真是        太        放肆        了

推荐        就是        有        这样        的

白素贞        水        漫        文水        城

我知道        那些夏天        就像        你        一样        回        不

渑池        至        洛阳        最早        的        车        几        点        哪里        坐        到        洛阳        几点

宏观        方面        大        的        流动性        格局        虽无        明显        变化        但        眼下        地方        政府        债务        限

电工        行业        竞争        大        锦力        电器        有        优势

兄弟        啊                 影技        1        班        q        群        是        多少

你们        家乡        话        叫        什么

深深        的        孤独感        与        挫败        感        感觉        个人

一起去        旅游        吧

谁知道        四会        那里        有        修        打火机        的

[root@cq01-forum-rstree01.cq01.baidu.com faster-rnnlm]# pwd

/home/users/chenghuige/other/faster-rnnlm.debug/faster-rnnlm

  1. 统计词频建立vocabulary

void Vocabulary::BuildFromCorpus(const std::string& fpath, bool show_progress)

首先添加一个 </s>

AddWord(kEOSTag);
只是编号0

 
 

然后逐个添加每行

每行处理的时候按照IsSpace切分

inline bool IsSpace(char c) {

return c == ' ' || c == '\r' || c == '\t' || c == '\n';

 
 

然后其实就是对每个词
类似 Identifer.h那样顺序编号,没出现的词
叫做oov 编号 -1

 
 

除了编号之外
同时统计频次

最后按照频次排序
从大到小
同时更新编号
也就是频次最大的
这里 </s> 编号为0

(gdb) p words_

, word = 0x6ae1c0 "</s>"}, {

freq = 258246, word = 0x6aef20 "\265\304"}, {freq = 126910, word = 0x6aeff0 "\301\313"}, {

freq = 101904, word = 0x6aedc0 "\316\322"}, {freq = 67328, word = 0x6aeee0 "\323\320"}, {

freq = 62290, word = 0x6aec10 "\270\366"}, {freq = 60866, word = 0x6afb20 "\322\273"}, {

 
 

[root@cq01-forum-rstree01.cq01.baidu.com faster-rnnlm]# wc -l thread.title.good.train.txt

thread.title.good.train.txt

 
 

gdb) p cfg

$2 = {layer_size = 5, layer_count = 1, maxent_hash_size = 199947228, maxent_order = 3, use_nce = false, nce_lnz = 9, reverse_sentence = false, layer_type = "sigmoid"}

 
 

  1. 构建网格结构

main_nnet = new NNet(vocab, cfg, use_cuda, use_cuda_memory_efficient);

构造函数调用Init 在这里

 
 

embeddings.resize(vocab.size(), cfg.layer_size);

//(word_num, hidden_size) 二维数组

 
 

rec_layer = CreateLayer(cfg.layer_type, cfg.layer_size, cfg.layer_count);

//隐层
建立一个layer 默认layer_type是sigmoid

 
 

maxent_layer.Init(cfg.maxent_hash_size);

//最大熵
@TODO

 
 

softmax_layer = HSTree::CreateHuffmanTree(vocab, cfg.layer_size);

//输出层 softmax 采用huffman树

 
 

 
 

Faster-rnnlm代码分析1 - 词表构建,Nnet成员的更多相关文章

  1. 完整全面的Java资源库(包括构建、操作、代码分析、编译器、数据库、社区等等)

    构建 这里搜集了用来构建应用程序的工具. Apache Maven:Maven使用声明进行构建并进行依赖管理,偏向于使用约定而不是配置进行构建.Maven优于Apache Ant.后者采用了一种过程化 ...

  2. Android代码分析工具lint学习

    1 lint简介 1.1 概述 lint是随Android SDK自带的一个静态代码分析工具.它用来对Android工程的源文件进行检查,找出在正确性.安全.性能.可使用性.可访问性及国际化等方面可能 ...

  3. 常用 Java 静态代码分析工具的分析与比较

    常用 Java 静态代码分析工具的分析与比较 简介: 本文首先介绍了静态代码分析的基 本概念及主要技术,随后分别介绍了现有 4 种主流 Java 静态代码分析工具 (Checkstyle,FindBu ...

  4. angular代码分析之异常日志设计

    angular代码分析之异常日志设计 错误异常是面向对象开发中的记录提示程序执行问题的一种重要机制,在程序执行发生问题的条件下,异常会在中断程序执行,同时会沿着代码的执行路径一步一步的向上抛出异常,最 ...

  5. [Asp.net 5] DependencyInjection项目代码分析4-微软的实现(3)

    这个系列已经写了5篇,链接地址如下: [Asp.net 5] DependencyInjection项目代码分析 [Asp.net 5] DependencyInjection项目代码分析2-Auto ...

  6. wifi display代码 分析

    转自:http://blog.csdn.net/lilian0118/article/details/23168531 这一章中我们来看Wifi Display连接过程的建立,包含P2P的部分和RTS ...

  7. Device Tree(三):代码分析【转】

    转自:http://www.wowotech.net/linux_kenrel/dt-code-analysis.html Device Tree(三):代码分析 作者:linuxer 发布于:201 ...

  8. 集成TFS Build生成与SonarQube获取代码分析结果

    软件项目在开发过程中,往往由于任务重.时间紧等原因忽略软件代码的质量和规范检查,只注重软件功能的开发和交付.等软件交付上线以后,由于代码质量导致的问题会耗费开发和运维人员的大量时间.研发表明,项目上线 ...

  9. Device Tree(三):代码分析

    一.前言 Device Tree总共有三篇,分别是: 1.为何要引入Device Tree,这个机制是用来解决什么问题的?(请参考引入Device Tree的原因) 2.Device Tree的基础概 ...

随机推荐

  1. MYSQL数据库导入出错:#1046 - No database selected

    今天遇到的mysql导入Navivat for MySql,总是出错,搞了一会才记起没有创建同名的数据库,然后还是导不进去,原来是要在建立的同名的数据单击右键---->运行Sql文件--> ...

  2. javascript中运算符的优先级

    运算符优先级 JavaScript中的运算符优先级是一套规则.该规则在计算表达式时控制运算符执行的顺序.具有较高优先级的运算符先于较低优先级的运算符执行.例如,乘法的执行先于加法. 下表按从最高到最低 ...

  3. 文件系统权限引起IIS站点总跳登录页面

    今天在IIS上部署一个ASP.NET站点时遇到一个很奇怪的问题,不管访问什么页面,都会跳到登录页面,即使是访问静态文件. 折腾半天,百思不得其解,百整不得其果... 后来突然想到,是不是站点所在文件夹 ...

  4. ASP.NET MVC必知必会知识点总结(二)

    一.实现Controller的依赖注入: 1.自定义继承DefaultControllerFactory 类的控制器工厂类并重写GetControllerInstance方法:(如:InjectCon ...

  5. elk系列1之入门安装与基本操作

    preface 我们每天都要查看服务器的日志,一方面是为了开发的同事翻找日志,另一方面是巡检服务器查看日志,而随着服务器数量以及越来越多的业务上线,日志越来越多,人肉运维相当痛苦了,此时,参考现在非常 ...

  6. Emgu 决策树

    MCvDTreeParams cvFolds //If this parameter is >1, the tree is pruned using cv_folds-fold cross va ...

  7. JS开发HTML5游戏《神奇的六边形》(三)

    近期出现一款魔性的消除类HTML5游戏<神奇的六边形>,今天我们一起来看看如何通过开源免费的青瓷引擎(www.zuoyouxi.com)来实现这款游戏. (点击图片可进入游戏体验) 因内容 ...

  8. C#二进制文件的读写

    sing System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using ...

  9. 杂记(编程style)----google code style!

    1.文件名 使用小写字母和下划线组合.头文件以.h结尾,定义文件用.cc结尾.例如:my_useful_class.cc 2.类型名 使用大写字母开头,多个单词组合时每个单词的首字母大写.例如:Url ...

  10. IOS NSInvocation用法简介

    IOS NSInvocation用法简介 2012-10-25 19:59 来源:博客园 作者:csj007523 字号:T|T [摘要]在 iOS中可以直接调用某个对象的消息方式有两种,其中一种就是 ...