首先
实现一个尽可能少调用tf.nn模块儿的,自己手写相关的function

 
 

import tensorflow as tf

import numpy as np

import melt_dataset

import sys

from sklearn.metrics import roc_auc_score

 
 

def init_weights(shape):

return tf.Variable(tf.random_normal(shape, stddev=0.01))

 
 

def model(X, w):

return 1.0/(1.0 + tf.exp(-(tf.matmul(X, w))))
#sigmoid

 
 

batch_size = 500

learning_rate = 0.001

num_iters = 1020

 
 

argv = sys.argv

trainset = argv[1]

testset = argv[2]

 
 

trX, trY = melt_dataset.load_dense_data(trainset)

print "finish loading train set ",trainset

teX, teY = melt_dataset.load_dense_data(testset)

print "finish loading test set ", testset

 
 

num_features = trX[0].shape[0]

print 'num_features: ',num_features

print 'trainSet size: ', len(trX)

print 'testSet size: ', len(teX)

print 'batch_size:', batch_size, ' learning_rate:', learning_rate, ' num_iters:', num_iters

 
 

X = tf.placeholder("float", [None, num_features]) # create symbolic variables

Y = tf.placeholder("float", [None, 1])

 
 

w = init_weights([num_features, 1]) # like in linear regression, we need a shared variable weight matrix for logistic regression

 
 

py_x = model(X, w)

 
 

cost = -tf.reduce_sum(Y*tf.log(py_x) + (1 - Y) * tf.log(1 - py_x))

train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) # construct optimizer

 
 

predict_op = py_x

 
 

sess = tf.Session()

init = tf.initialize_all_variables()

sess.run(init)

 
 

for i in range(num_iters):

predicts, cost_ = sess.run([predict_op, cost], feed_dict={X: teX, Y: teY})

print i, 'auc:', roc_auc_score(teY, predicts), 'cost:', cost_

for start, end in zip(range(0, len(trX), batch_size), range(batch_size, len(trX), batch_size)):

sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})

 
 

predicts, cost_ = sess.run([predict_op, cost], feed_dict={X: teX, Y: teY})

print 'final ', 'auc:', roc_auc_score(teY, predicts),'cost:', cost_

 
 

注意如果设置的batch_size 比较大
而learning rate也比较大
可能会出现nan,
可以通过减小batch_size

或者调小learning rate来避免

 
 

更好的方式是使用tensorflow自带的函数

tf.nn.sigmoid_cross_entropy_with_logits(logits, targets, name=None)

 
 

Computes sigmoid cross entropy given logits.

 
 

Measures the probability error in discrete classification tasks in which each class is independent and not mutually exclusive. For instance, one could perform multilabel classification where a picture can contain both an elephant and a dog at the same time.

 
 

For brevity, let x = logits, z = targets. The logistic loss is

原始的logistic误差,注意和
-tf.reduce_sum(Y*tf.log(py_x) + (1 - Y) * tf.log(1 - py_x))
是等价的

x - x * z + log(1 + exp(-x))

 
 

Z = 1 , log(1 + exp(-x))

Z = 0 , x + log(1 + exp(-x) ) = log(1 + exp(x))

To ensure stability and avoid overflow, the implementation uses

为了避免溢出。。nan的产生。。

 
 

max(x, 0) - x * z + log(1 + exp(-abs(x)))

 
 

如果z = 1, x >= 0 和原始一致

如果z = 1, x < 0 那么
-x + log(1 + exp(x)) = log(1+ exp(x) / exp(x)) = log(1 + exp(-x)) 还是一样。。

如果z = 0, x <= 0 log(1 + exp(x))

如果 z = 0, x > 0 x + log(1 + exp(-x)) = log(1 + exp(x))

 
 

感觉就是避免了 exp(x) 而 x过大? @TODO

logits and targets must have the same type and shape.

注意尽管采用这个避免了nan产生
但是实际看
过程的话 auc 会迭代中变化不稳定
感觉还是调整下learning rate比较好。

 
 

最后tf.nn.sigmoid也可以替代手写的

y = 1 / (1 + exp(-x)).

 
 

来自 <http://www.tensorflow.org/api_docs/python/nn.md#sigmoid>

 
 

最终版本

import tensorflow as tf

import numpy as np

import melt_dataset

import sys

from sklearn.metrics import roc_auc_score

 
 

def init_weights(shape):

return tf.Variable(tf.random_normal(shape, stddev=0.01))

 
 

def model(X, w):

return tf.matmul(X,w)

 
 

batch_size = 500

learning_rate = 0.001

num_iters = 120

 
 

argv = sys.argv

trainset = argv[1]

testset = argv[2]

 
 

trX, trY = melt_dataset.load_dense_data(trainset)

print "finish loading train set ",trainset

teX, teY = melt_dataset.load_dense_data(testset)

print "finish loading test set ", testset

 
 

num_features = trX[0].shape[0]

print 'num_features: ',num_features

print 'trainSet size: ', len(trX)

print 'testSet size: ', len(teX)

print 'batch_size:', batch_size, ' learning_rate:', learning_rate, ' num_iters:', num_iters

 
 

X = tf.placeholder("float", [None, num_features]) # create symbolic variables

Y = tf.placeholder("float", [None, 1])

 
 

w = init_weights([num_features, 1]) # like in linear regression, we need a shared variable weight matrix for logistic regression

 
 

py_x = model(X, w)

 
 

cost = tf.reduce_sum(tf.nn.sigmoid_cross_entropy_with_logits(py_x, Y))

train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) # construct optimizer

predict_op = tf.nn.sigmoid(py_x)

 
 

sess = tf.Session()

init = tf.initialize_all_variables()

sess.run(init)

 
 

for i in range(num_iters):

predicts, cost_ = sess.run([predict_op, cost], feed_dict={X: teX, Y: teY})

print i, 'auc:', roc_auc_score(teY, predicts), 'cost:', cost_

for start, end in zip(range(0, len(trX), batch_size), range(batch_size, len(trX), batch_size)):

sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})

 
 

predicts, cost_ = sess.run([predict_op, cost], feed_dict={X: teX, Y: teY})

print 'final ', 'auc:', roc_auc_score(teY, predicts),'cost:', cost_

 
 

 
 

运行结果

./logistic_regression.py corpus/feature.normed.rand.12000.0_2.txt corpus/feature.normed.rand.12000.1_2.txt

/home/users/chenghuige/.jumbo/lib/python2.7/site-packages/sklearn/externals/joblib/_multiprocessing_helpers.py:29: UserWarning: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.. joblib will operate in serial mode

warnings.warn('%s. joblib will operate in serial mode' % (e,))

... loading data: corpus/feature.normed.rand.12000.0_2.txt

10000

finish loading train set corpus/feature.normed.rand.12000.0_2.txt

... loading data: corpus/feature.normed.rand.12000.1_2.txt

finish loading test set corpus/feature.normed.rand.12000.1_2.txt

num_features: 493

trainSet size: 10001

testSet size: 1999

batch_size: 500 learning_rate: 0.001 num_iters: 120

I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 24

I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 24

0 auc: 0.380243958856 cost: 1350.72

1 auc: 0.876460425134 cost: 538.122

2 auc: 0.894349010333 cost: 493.974

3 auc: 0.900608480969 cost: 479.184

4 auc: 0.904222074311 cost: 471.299

5 auc: 0.906476144619 cost: 466.076

6 auc: 0.908203871808 cost: 462.155

7 auc: 0.909598799088 cost: 458.986

8 auc: 0.910892234197 cost: 456.31

9 auc: 0.911966162982 cost: 453.987

10 auc: 0.912926798182 cost: 451.936

11 auc: 0.913835507154 cost: 450.105

12 auc: 0.91452234952 cost: 448.458

13 auc: 0.915244596132 cost: 446.968

14 auc: 0.915910195951 cost: 445.613

15 auc: 0.916483744731 cost: 444.375

16 auc: 0.916939279358 cost: 443.243

17 auc: 0.917430218232 cost: 442.202

18 auc: 0.917977803898 cost: 441.244

19 auc: 0.918386132865 cost: 440.36

20 auc: 0.918782660417 cost: 439.541

21 auc: 0.919139063156 cost: 438.782

22 auc: 0.919471863066 cost: 438.076

23 auc: 0.91976925873 cost: 437.418

24 auc: 0.920017088449 cost: 436.803

25 auc: 0.920342807509 cost: 436.229

26 auc: 0.920616600343 cost: 435.69

27 auc: 0.920784180439 cost: 435.185

28 auc: 0.920965922233 cost: 434.709

29 auc: 0.921171266858 cost: 434.261

30 auc: 0.921402574597 cost: 433.839

31 auc: 0.921548912146 cost: 433.439

32 auc: 0.921770778752 cost: 433.061

33 auc: 0.921959601395 cost: 432.703

34 auc: 0.922181468002 cost: 432.363

35 auc: 0.922372650928 cost: 432.04

36 auc: 0.922474143099 cost: 431.734

37 auc: 0.922618120365 cost: 431.441

38 auc: 0.92279278131 cost: 431.163

39 auc: 0.922929677727 cost: 430.897

40 auc: 0.92300992735 cost: 430.643

41 auc: 0.923146823767 cost: 430.401

42 auc: 0.923253036504 cost: 430.169

43 auc: 0.92331204358 cost: 429.947

44 auc: 0.923441859148 cost: 429.735

45 auc: 0.923557513017 cost: 429.531

46 auc: 0.923649564056 cost: 429.336

47 auc: 0.923741615094 cost: 429.148

48 auc: 0.923826585284 cost: 428.968

49 auc: 0.923897393775 cost: 428.795

50 auc: 0.923991805097 cost: 428.629

51 auc: 0.924067334155 cost: 428.469

52 auc: 0.924131061797 cost: 428.315

53 auc: 0.924161745477 cost: 428.166

54 auc: 0.924220752553 cost: 428.024

55 auc: 0.924251436232 cost: 427.886

56 auc: 0.92429628161 cost: 427.754

57 auc: 0.924355288686 cost: 427.627

58 auc: 0.924440258876 cost: 427.504

59 auc: 0.924515787933 cost: 427.385

60 auc: 0.924560633311 cost: 427.27

61 auc: 0.92465268435 cost: 427.16

62 auc: 0.924699890011 cost: 427.053

63 auc: 0.924780139634 cost: 426.95

64 auc: 0.924834426144 cost: 426.851

65 auc: 0.924829705578 cost: 426.755

66 auc: 0.924867470107 cost: 426.663

67 auc: 0.924891072937 cost: 426.573

68 auc: 0.924907594919 cost: 426.487

69 auc: 0.924950080014 cost: 426.404

70 auc: 0.924961881429 cost: 426.323

71 auc: 0.925023248788 cost: 426.245

72 auc: 0.925051572185 cost: 426.17

73 auc: 0.925079895581 cost: 426.097

74 auc: 0.925120020393 cost: 426.027

75 auc: 0.92517194662 cost: 425.959

76 auc: 0.925240394828 cost: 425.894

77 auc: 0.925294681338 cost: 425.83

78 auc: 0.925330085584 cost: 425.769

79 auc: 0.925391452943 cost: 425.71

80 auc: 0.925410335207 cost: 425.653

81 auc: 0.925436298321 cost: 425.598

82 auc: 0.925485864265 cost: 425.544

83 auc: 0.925500025963 cost: 425.493

84 auc: 0.925530709643 cost: 425.443

85 auc: 0.92555195219 cost: 425.395

86 auc: 0.925594437285 cost: 425.349

87 auc: 0.92563692238 cost: 425.304

88 auc: 0.925665245776 cost: 425.261

89 auc: 0.925703010305 cost: 425.219

90 auc: 0.925721892569 cost: 425.179

91 auc: 0.925771458513 cost: 425.14

92 auc: 0.925778539362 cost: 425.103

93 auc: 0.925778539362 cost: 425.066

94 auc: 0.925771458513 cost: 425.032

95 auc: 0.925780899645 cost: 424.998

96 auc: 0.925809223042 cost: 424.966

97 auc: 0.925806862759 cost: 424.935

98 auc: 0.925835186156 cost: 424.905

99 auc: 0.925851708137 cost: 424.876

100 auc: 0.925868230118 cost: 424.848

101 auc: 0.925863509552 cost: 424.821

102 auc: 0.925887112383 cost: 424.795

103 auc: 0.925905994647 cost: 424.77

104 auc: 0.925922516628 cost: 424.747

105 auc: 0.925915435779 cost: 424.724

106 auc: 0.925934318043 cost: 424.702

107 auc: 0.925920156345 cost: 424.681

108 auc: 0.925965001723 cost: 424.661

109 auc: 0.925955560591 cost: 424.642

110 auc: 0.926005126535 cost: 424.623

111 auc: 0.926026369082 cost: 424.605

112 auc: 0.926035810214 cost: 424.588

113 auc: 0.926009847101 cost: 424.572

114 auc: 0.926000405969 cost: 424.557

115 auc: 0.926021648516 cost: 424.542

116 auc: 0.92604053078 cost: 424.528

117 auc: 0.926057052762 cost: 424.514

118 auc: 0.926075935026 cost: 424.501

119 auc: 0.926090096724 cost: 424.489

final auc: 0.926087736441 cost: 424.478

 
 

程序另外一个需要注意的是 Y,label 需要

[batch_size, 1]这样

[

[0],

[1],

[0],

]

不能是[0,1,0…]也就是[batch_size,]是不行的

 
 

看一下LinearSVM和gbdt的结果

mlt -c tt ./corpus/feature.normed.rand.12000.0_2.txt ./corpus/feature.normed.rand.12000.1_2.tx

Confusion table:

||===============================||

|| PREDICTED ||

TRUTH || positive | negative || RECALL

||===============================||

positive|| 676 | 532 || 0.5596 (676/1208)

negative|| 192 | 8601 || 0.9782 (8601/8793)

||===============================||

PRECISION 0.7788 (676/868) 0.9417(8601/9133)

LOG-LOSS/instance:                0.2412

LOG-LOSS-PROB/instance:                0.1912

TEST-SET ENTROPY (prior LL/in):        0.3685

LOG-LOSS REDUCTION (RIG):        48.1082%

 
 

OVERALL 0/1 ACCURACY:        0.9276 (9277/10001)

POS.PRECISION:                0.7788

POS.RECALL:                0.5596

NEG.PRECISION:                0.9417

NEG.RECALL:                0.9782

F1.SCORE:                 0.6513

OuputAUC: 0.9309

AUC: [0.9309]

 
 

mlt -c tt ./corpus/feature.normed.rand.12000.0_2.txt ./corpus/feature.normed.rand.12000.1_2.txt -cl gbdt

Confusion table:

||===============================||

|| PREDICTED ||

TRUTH || positive | negative || RECALL

||===============================||

positive|| 1194 | 14 || 0.9884 (1194/1208)

negative|| 0 | 8793 || 1.0000 (8793/8793)

||===============================||

PRECISION 1.0000 (1194/1194) 0.9984(8793/8807)

LOG-LOSS/instance:                0.0214

LOG-LOSS-PROB/instance:                0.0097

TEST-SET ENTROPY (prior LL/in):        0.3685

LOG-LOSS REDUCTION (RIG):        97.3625%

 
 

OVERALL 0/1 ACCURACY:        0.9986 (9987/10001)

POS.PRECISION:                1.0000

POS.RECALL:                0.9884

NEG.PRECISION:                0.9984

NEG.RECALL:                1.0000

F1.SCORE:                 0.9942

OuputAUC: 0.9988

AUC: [0.9988]

 
 

 
 

Tensorflow 实现稠密输入数据的逻辑回归二分类的更多相关文章

  1. matlab-逻辑回归二分类(Logistic Regression)

    逻辑回归二分类 今天尝试写了一下逻辑回归分类,把代码分享给大家,至于原理的的话请戳这里 https://blog.csdn.net/laobai1015/article/details/7811321 ...

  2. 深度学习原理与框架-Tensorflow基本操作-mnist数据集的逻辑回归 1.tf.matmul(点乘操作) 2.tf.equal(对应位置是否相等) 3.tf.cast(将布尔类型转换为数值类型) 4.tf.argmax(返回最大值的索引) 5.tf.nn.softmax(计算softmax概率值) 6.tf.train.GradientDescentOptimizer(损失值梯度下降器)

    1. tf.matmul(X, w) # 进行点乘操作 参数说明:X,w都表示输入的数据, 2.tf.equal(x, y) # 比较两个数据对应位置的数是否相等,返回值为True,或者False 参 ...

  3. 【R语言学习笔记】 Day1 CART 逻辑回归、分类树以及随机森林的应用及对比

    1. 目的:根据人口普查数据来预测收入(预测每个个体年收入是否超过$50,000) 2. 数据来源:1994年美国人口普查数据,数据中共含31978个观测值,每个观测值代表一个个体 3. 变量介绍: ...

  4. Logistic回归二分类Winner or Losser----台大李宏毅机器学习作业二(HW2)

    一.作业说明 给定训练集spam_train.csv,要求根据每个ID各种属性值来判断该ID对应角色是Winner还是Losser(0.1分类). 训练集介绍: (1)CSV文件,大小为4000行X5 ...

  5. TensorFlow学习笔记2:逻辑回归实现手写字符识别

    代码比较简单,没啥好说的,就做个记录而已.大致就是现建立graph,再通过session运行即可.需要注意的就是Variable要先初始化再使用. import tensorflow as tf fr ...

  6. Tensorflow学习教程------实现lenet并且进行二分类

    #coding:utf-8 import tensorflow as tf import os def read_and_decode(filename): #根据文件名生成一个队列 filename ...

  7. 通俗地说逻辑回归【Logistic regression】算法(二)sklearn逻辑回归实战

    前情提要: 通俗地说逻辑回归[Logistic regression]算法(一) 逻辑回归模型原理介绍 上一篇主要介绍了逻辑回归中,相对理论化的知识,这次主要是对上篇做一点点补充,以及介绍sklear ...

  8. 逻辑回归(Logistic Regression)二分类原理及python实现

    本文目录: 1. sigmoid function (logistic function) 2. 逻辑回归二分类模型 3. 神经网络做二分类问题 4. python实现神经网络做二分类问题 1. si ...

  9. 大白话5分钟带你走进人工智能-第二十节逻辑回归和Softmax多分类问题(5)

                                                        大白话5分钟带你走进人工智能-第二十节逻辑回归和Softmax多分类问题(5) 上一节中,我们讲 ...

随机推荐

  1. Hawk 4.6 并行化

    并行化 Hawk支持单机并行化,也就是使用多线程获取数据.它可以控制目前所有任务的数量,为了不给网站造成过大的压力,仅当任务池中的任务数量小于一定值后,才会插入新的任务. 你可以在数据清洗的 执行面板 ...

  2. flex 布局笔记

    1,今天遇到一个问题,就是当元素布局设置为了flex后,里面的内容只有文字,但是对text-align 属性设置无效,仔细想了下,是因为把display 设置为了flex后,flex将里面的文字也认为 ...

  3. apache限制同一ip大并发访问及下载

    内容来自以下网站整理 http://www.jb51.net/article/58060.htmhttp://www.2cto.com/os/201202/120665.htmlhttp://blog ...

  4. AES加密解密

    AES简介 美国国家标准技术研究所在2001年发布了高级加密标准(AES).AES是一个对称分组密码算法,旨在取代DES成为广泛使用的标准. 根据使用的密码长度,AES最常见的有3种密钥方案,用以适应 ...

  5. Jquery Offset, Document, Window 都是什么

    From http://www.cnblogs.com/luhe/archive/2012/11/14/2769263.html   JQuery Offset实验与应用 我们有时候需要实现这样一种功 ...

  6. 前端面试送命题(二)-callback,promise,generator,async-await

    前言 本篇文章适合前端架构师,或者进阶的前端开发人员:我在面试vmware前端架构师的时候,被问到关于callback,promise,generator,async-await的问题. 首先我们回顾 ...

  7. NodeJS 难点(网络,文件)的 核心 stream 三:readable ?

    什么是可读流 可读流    常见  读取磁盘文件.读取网络请求内容等,看一下前面介绍什么是流用的例子: const rs = fs.createReadStream(filePath); 我们常见的控 ...

  8. 一个简单的nodejs项目(cat-names)分析

    https://github.com/sindresorhus/cat-names 一个非常简单的nodejs项目,用来方便的获取猫猫的名字: 安装: npm install --save cat-n ...

  9. 【算法学习】有旋treap

    treap是平衡树的一种.与其他平衡树一样,它也能够支持插入和删除,求第k极值等,接下来我们主要探讨有旋treap的实现过程. treap中每个节点要维护其值,左右孩子以及子树大小.父亲要不要写则看你 ...

  10. Android-bindService远程服务(Aidl)-初步

    之前上一篇讲解到本地服务,本地服务只能在自身APP中Activity访问Service,调用Service里面到方法等操作 如果想A应用访问B应用里面的方法,属于跨进程调用,如果Android不特供这 ...